[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:10 dist-test-slave-68xk krb5kdc[31344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:10 dist-test-slave-68xk krb5kdc[31344](info): set up 2 sockets
Aug 10 18:53:10 dist-test-slave-68xk krb5kdc[31344](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 18:53:12.083509 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:12 dist-test-slave-68xk krb5kdc[31344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:12.218516 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42967
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:42967
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:12.353420 31364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:12.353760 31364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:12.353838 31364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:12.357892 31364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:12.357990 31364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:12.358027 31364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:12.358076 31364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:12.358115 31364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:12.363648 31364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:42967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:12.364764 31364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:12.365881 31364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:12.375293 31376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:12.375288 31374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:12.375336 31364 server_base.cc:1034] running on GCE node
W20260810 18:53:12.375293 31372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:12.376240 31364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:12.377689 31364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:12.378901 31364 hybrid_clock.cc:648] HybridClock initialized: now 1786387992378886 us; error 39 us; skew 500 ppm
Aug 10 18:53:12 dist-test-slave-68xk krb5kdc[31344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:12.382046 31364 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:12.383000 31364 webserver.cc:460] Webserver started at http://127.30.153.190:34977/ using document root <none> and password file <none>
I20260810 18:53:12.383556 31364 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:12.383621 31364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:12.383811 31364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:12.386107 31364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "0ae1e007c6de49e78076bc40939e26af"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "eaa29704df3bf577f2e6d80b0ddc10de"
server_key_iv: "d47616dcc126b14e4e137734e0edcdc5"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.386771 31364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "0ae1e007c6de49e78076bc40939e26af"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "eaa29704df3bf577f2e6d80b0ddc10de"
server_key_iv: "d47616dcc126b14e4e137734e0edcdc5"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.391108 31364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:53:12.393987 31385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:12.395373 31364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:12.395500 31364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "0ae1e007c6de49e78076bc40939e26af"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "eaa29704df3bf577f2e6d80b0ddc10de"
server_key_iv: "d47616dcc126b14e4e137734e0edcdc5"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.395596 31364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:12.411031 31364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:12.413916 31364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:12.414121 31364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:12.424999 31364 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:42967
I20260810 18:53:12.425153 31474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:42967 every 8 connection(s)
I20260810 18:53:12.426205 31364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:12.429250 31476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:12.435681 31476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af: Bootstrap starting.
I20260810 18:53:12.436153 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31364
I20260810 18:53:12.436333 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:12.438066 31334 external_mini_cluster.cc:1428] Setting key c088bd2ef511df5dd8ccf22127f63af4
I20260810 18:53:12.439405 31476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:12.440547 31476 log.cc:824] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:12.443218 31476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af: No bootstrap required, opened a new log
I20260810 18:53:12.447793 31476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae1e007c6de49e78076bc40939e26af" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42967 } }
I20260810 18:53:12.448077 31476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:12.448143 31476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae1e007c6de49e78076bc40939e26af, State: Initialized, Role: FOLLOWER
I20260810 18:53:12.448626 31476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae1e007c6de49e78076bc40939e26af" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42967 } }
I20260810 18:53:12.448815 31476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:12.448915 31476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:12.449038 31476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:12.450284 31476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae1e007c6de49e78076bc40939e26af" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42967 } }
I20260810 18:53:12.450726 31476 leader_election.cc:302] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ae1e007c6de49e78076bc40939e26af; no voters:
I20260810 18:53:12.451064 31476 leader_election.cc:288] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:12.451438 31484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:12.451792 31484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [term 1 LEADER]: Becoming Leader. State: Replica: 0ae1e007c6de49e78076bc40939e26af, State: Running, Role: LEADER
I20260810 18:53:12.452162 31484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae1e007c6de49e78076bc40939e26af" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42967 } }
I20260810 18:53:12.452425 31476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:12.454545 31489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ae1e007c6de49e78076bc40939e26af. Latest consensus state: current_term: 1 leader_uuid: "0ae1e007c6de49e78076bc40939e26af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae1e007c6de49e78076bc40939e26af" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42967 } } }
I20260810 18:53:12.454733 31489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:12.455179 31486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ae1e007c6de49e78076bc40939e26af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae1e007c6de49e78076bc40939e26af" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42967 } } }
I20260810 18:53:12.455304 31486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af [sys.catalog]: This master's current role is: LEADER
Aug 10 18:53:12 dist-test-slave-68xk krb5kdc[31344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
W20260810 18:53:12.460760 31505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:12.460855 31505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:12.460989 31504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:12.462476 31504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:12.469941 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:12.441943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.30.153.190:42967)
0810 18:53:12.442499 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:12.442557 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:12.444039 (+ 1482us) server_negotiation.cc:407] Connection header received
0810 18:53:12.447268 (+ 3229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:12.447297 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:12.447667 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:12.448029 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:12.450609 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:12.452150 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:12.454186 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:12.455140 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:12.455360 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:12.461787 (+ 6427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:12.461795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:12.461825 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:12.461869 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:12.464910 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:12.465730 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:12.465753 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:12.465760 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:12.465930 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:12.466514 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:12.466518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:12.466520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:12.467251 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:12.467487 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:12.468218 (+ 731us) server_negotiation.cc:299] Negotiation successful
0810 18:53:12.468549 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":110,"threads_started":1}
I20260810 18:53:12.470944 31504 catalog_manager.cc:1347] Generated new cluster ID: 89cf696a470a4beeaa460b5abdfb3298
I20260810 18:53:12.471009 31504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:12.500552 31504 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:12.501631 31504 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:12.509932 31504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ae1e007c6de49e78076bc40939e26af: Generated new TSK 0
I20260810 18:53:12.510756 31504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:12.543390 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42967
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:12.678704 31512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:12.679123 31512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:12.679221 31512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:12.683483 31512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:12.683833 31512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:12.684018 31512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:12.690222 31512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:12.691404 31512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:12.692418 31512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:53:12.705106 31512 server_base.cc:1034] running on GCE node
W20260810 18:53:12.705062 31526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:12.704974 31522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:12.705056 31521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:12.705772 31512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:12.706513 31512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:12.707713 31512 hybrid_clock.cc:648] HybridClock initialized: now 1786387992707700 us; error 34 us; skew 500 ppm
Aug 10 18:53:12 dist-test-slave-68xk krb5kdc[31344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:12.711108 31512 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:12.711990 31512 webserver.cc:460] Webserver started at http://127.30.153.129:41897/ using document root <none> and password file <none>
I20260810 18:53:12.712574 31512 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:12.712677 31512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:12.712913 31512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:12.715231 31512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "d1617dc3165a44289eff629239e89c56"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "085da7620c0b08c95896bae8675954fb"
server_key_iv: "464dac6ea58126e576c5e2c9205ce01e"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.715826 31512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "d1617dc3165a44289eff629239e89c56"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "085da7620c0b08c95896bae8675954fb"
server_key_iv: "464dac6ea58126e576c5e2c9205ce01e"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.720068 31512 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:12.722697 31535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:12.723980 31512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:53:12.724131 31512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "d1617dc3165a44289eff629239e89c56"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "085da7620c0b08c95896bae8675954fb"
server_key_iv: "464dac6ea58126e576c5e2c9205ce01e"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.724249 31512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:12.743359 31512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:12.746769 31512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:12.747030 31512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:12.747927 31512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:12.749094 31512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:12.749146 31512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:12.749194 31512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:12.749212 31512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:12.767051 31512 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:36653
I20260810 18:53:12.767357 31707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:36653 every 8 connection(s)
I20260810 18:53:12.768042 31512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:12.770699 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31512
I20260810 18:53:12.770838 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:12.771147 31334 external_mini_cluster.cc:1428] Setting key 22778d48262122e372bc90c24d737ed1
Aug 10 18:53:12 dist-test-slave-68xk krb5kdc[31344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:53:12.786006 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:12.770516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:47041 (local address 127.30.153.190:42967)
0810 18:53:12.770680 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:12.770684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:12.772429 (+ 1745us) server_negotiation.cc:407] Connection header received
0810 18:53:12.774019 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:12.774022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:12.774074 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:12.774135 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:12.776108 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:12.776649 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:12.777936 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:12.778242 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:12.778359 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:12.781643 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:12.781647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:12.781649 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:12.781679 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:12.783406 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:12.783957 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:12.783965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:12.783967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:12.784049 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:12.784622 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:12.784625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:12.784627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:12.784832 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:12.784946 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:12.785608 (+ 662us) server_negotiation.cc:299] Negotiation successful
0810 18:53:12.785767 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
I20260810 18:53:12.787317 31711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:12.770860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42967 (local address 127.30.153.129:47041)
0810 18:53:12.772257 (+ 1397us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:12.772313 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:12.773774 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:12.774332 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:12.774343 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:12.775110 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:12.775936 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:12.775954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:12.776792 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:12.776795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:12.777789 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:12.777799 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:12.778400 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:12.778404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:12.778657 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:12.779367 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:12.779384 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:12.781453 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:12.783532 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:12.783538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:12.783554 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:12.783838 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:12.784339 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:12.784342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:12.784343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:12.784499 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:12.784946 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:12.784954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:12.785264 (+ 310us) client_negotiation.cc:769] Sending connection context
0810 18:53:12.785539 (+ 275us) client_negotiation.cc:240] Negotiation successful
0810 18:53:12.785912 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1149,"thread_start_us":878,"threads_started":1}
I20260810 18:53:12.788820 31708 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42967
I20260810 18:53:12.789093 31708 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:12.789620 31708 heartbeater.cc:511] Master 127.30.153.190:42967 requested a full tablet report, sending...
I20260810 18:53:12.792178 31414 ts_manager.cc:194] Registered new tserver with Master: d1617dc3165a44289eff629239e89c56 (127.30.153.129:36653)
I20260810 18:53:12.794459 31414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:47041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:12.806171 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42967
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:12.943183 31717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:12.943455 31717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:12.943542 31717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:12.947811 31717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:12.947871 31717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:12.947964 31717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:12.953712 31717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:12.954901 31717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:12.956027 31717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:12.966459 31723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:12.966509 31728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:12.966475 31725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:12.966714 31717 server_base.cc:1034] running on GCE node
I20260810 18:53:12.967162 31717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:12.967850 31717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:12.969110 31717 hybrid_clock.cc:648] HybridClock initialized: now 1786387992969081 us; error 51 us; skew 500 ppm
Aug 10 18:53:12 dist-test-slave-68xk krb5kdc[31344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:12.972374 31717 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:12.973299 31717 webserver.cc:460] Webserver started at http://127.30.153.130:34375/ using document root <none> and password file <none>
I20260810 18:53:12.973876 31717 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:12.973940 31717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:12.974228 31717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:12.976459 31717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "e397630dc2854a0e8ddfd44046c3a9a4"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "c63e46427cbeb022855fc32bae3aa729"
server_key_iv: "249842891bf500a0da4a350c5808f4dd"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.977069 31717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "e397630dc2854a0e8ddfd44046c3a9a4"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "c63e46427cbeb022855fc32bae3aa729"
server_key_iv: "249842891bf500a0da4a350c5808f4dd"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.981222 31717 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260810 18:53:12.984251 31741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:12.985752 31717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:53:12.985924 31717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "e397630dc2854a0e8ddfd44046c3a9a4"
format_stamp: "Formatted at 2026-08-10 18:53:12 on dist-test-slave-68xk"
server_key: "c63e46427cbeb022855fc32bae3aa729"
server_key_iv: "249842891bf500a0da4a350c5808f4dd"
server_key_version: "encryptionkey@0"
I20260810 18:53:12.986063 31717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:13.003363 31717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:13.008760 31717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:13.009083 31717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:13.010174 31717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:13.011546 31717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:13.011620 31717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.011668 31717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:13.011693 31717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.031045 31717 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:33775
I20260810 18:53:13.031116 31928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:33775 every 8 connection(s)
I20260810 18:53:13.032155 31717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:13.034404 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31717
I20260810 18:53:13.034543 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:13.034947 31334 external_mini_cluster.cc:1428] Setting key ec146c6856949a08af75e90184108d03
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[31344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387992, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:13.056144 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.034228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:48533 (local address 127.30.153.190:42967)
0810 18:53:13.034381 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:13.034384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:13.036037 (+ 1653us) server_negotiation.cc:407] Connection header received
0810 18:53:13.038014 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:13.038017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:13.038075 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:13.038185 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:13.039714 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.040243 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.041498 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.041732 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.042012 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.051580 (+ 9568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:13.051587 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:13.051593 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:13.051633 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:13.053664 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.054385 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.054387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.054388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.054531 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.054947 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.054950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.054952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.055155 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:13.055296 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:13.055821 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 18:53:13.055963 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:13.056983 31932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.034970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42967 (local address 127.30.153.130:48533)
0810 18:53:13.035862 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:13.035910 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:13.037785 (+ 1875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:13.038370 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:13.038377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:13.038951 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:13.039548 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:13.039565 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.040407 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.040411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:13.041355 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:13.041365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.042045 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.042049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:13.042331 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.048233 (+ 5902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:13.048252 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:13.051336 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:13.053790 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:13.053799 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:13.053817 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:13.054267 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:13.054665 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:13.054674 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:13.054676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:13.054828 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:13.055294 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:13.055301 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:13.055566 (+ 265us) client_negotiation.cc:769] Sending connection context
0810 18:53:13.055765 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 18:53:13.056109 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":208,"threads_started":1}
I20260810 18:53:13.058238 31930 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42967
I20260810 18:53:13.058540 31930 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:13.059059 31930 heartbeater.cc:511] Master 127.30.153.190:42967 requested a full tablet report, sending...
I20260810 18:53:13.060160 31414 ts_manager.cc:194] Registered new tserver with Master: e397630dc2854a0e8ddfd44046c3a9a4 (127.30.153.130:33775)
I20260810 18:53:13.060823 31414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:48533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:13.070667 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42967
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:13.240685 31936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:13.240950 31936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:13.241027 31936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:13.245143 31936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:13.245198 31936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:13.245291 31936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:13.250780 31936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42967
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:13.251843 31936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:13.252933 31936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:13.264227 31945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:13.264318 31944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:13.264349 31947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:13.264556 31936 server_base.cc:1034] running on GCE node
I20260810 18:53:13.265019 31936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:13.265683 31936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:13.266878 31936 hybrid_clock.cc:648] HybridClock initialized: now 1786387993266869 us; error 34 us; skew 500 ppm
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[31344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:13.270215 31936 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:13.271076 31936 webserver.cc:460] Webserver started at http://127.30.153.131:41749/ using document root <none> and password file <none>
I20260810 18:53:13.271662 31936 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:13.271726 31936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:13.271945 31936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:13.274096 31936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "16bb0c201aae4a33a90f25f8e5b765a5"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-68xk"
server_key: "2898c314d171f2f9c990e36333ea87ca"
server_key_iv: "4b77eecec7d16724369f24589714c667"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.274721 31936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "16bb0c201aae4a33a90f25f8e5b765a5"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-68xk"
server_key: "2898c314d171f2f9c990e36333ea87ca"
server_key_iv: "4b77eecec7d16724369f24589714c667"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.278844 31936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:53:13.281534 31958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.283025 31936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:13.283154 31936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "16bb0c201aae4a33a90f25f8e5b765a5"
format_stamp: "Formatted at 2026-08-10 18:53:13 on dist-test-slave-68xk"
server_key: "2898c314d171f2f9c990e36333ea87ca"
server_key_iv: "4b77eecec7d16724369f24589714c667"
server_key_version: "encryptionkey@0"
I20260810 18:53:13.283339 31936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:13.302613 31936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:13.306250 31936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:13.306505 31936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:13.307149 31936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:13.308027 31936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:13.308100 31936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.308176 31936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:13.308223 31936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:13.324119 31936 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:43027
I20260810 18:53:13.324191 32145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:43027 every 8 connection(s)
I20260810 18:53:13.325167 31936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:13.330493 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31936
I20260810 18:53:13.330669 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:13.330962 31334 external_mini_cluster.cc:1428] Setting key 02b2e93efb5bd8d3e3bac94919c0ade0
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[31344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:13.340685 31481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.327575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:51305 (local address 127.30.153.190:42967)
0810 18:53:13.327778 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:13.327783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:13.329062 (+ 1279us) server_negotiation.cc:407] Connection header received
0810 18:53:13.330297 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:13.330301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:13.330360 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:13.330504 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:13.332035 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.332565 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.333736 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.333952 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.334062 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.336953 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:13.336957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:13.336959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:13.336982 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:13.338556 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.339082 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.339086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.339088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.339179 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.339555 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.339560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.339561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.339764 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:13.339879 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:13.340389 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 18:53:13.340514 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 18:53:13.341545 32148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.327979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42967 (local address 127.30.153.131:51305)
0810 18:53:13.328908 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:13.328950 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:13.330059 (+ 1109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:13.330692 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:13.330701 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:13.331277 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:13.331880 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:13.331895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.332699 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.332702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:13.333584 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:13.333593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.334112 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.334116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:13.334294 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.334939 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:13.334954 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:13.336782 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:13.338682 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:13.338688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:13.338702 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:13.338970 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:13.339299 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:13.339303 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:13.339305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:13.339451 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:13.339880 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:13.339888 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:13.340129 (+ 241us) client_negotiation.cc:769] Sending connection context
0810 18:53:13.340322 (+ 193us) client_negotiation.cc:240] Negotiation successful
0810 18:53:13.340647 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":224,"threads_started":1}
I20260810 18:53:13.342882 32146 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42967
I20260810 18:53:13.343148 32146 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:13.343657 32146 heartbeater.cc:511] Master 127.30.153.190:42967 requested a full tablet report, sending...
I20260810 18:53:13.345021 31413 ts_manager.cc:194] Registered new tserver with Master: 16bb0c201aae4a33a90f25f8e5b765a5 (127.30.153.131:43027)
I20260810 18:53:13.345819 31413 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:51305
I20260810 18:53:13.346032 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[31344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[31344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:13.380609 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.369131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.30.153.129:36653)
0810 18:53:13.369456 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:13.369467 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:13.369505 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:53:13.369794 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:13.369801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:13.369998 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:13.370198 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:13.371104 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.372083 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.373022 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.373505 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.373643 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.376607 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:13.376617 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:13.376630 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:13.376658 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:13.378620 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.379098 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.379103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.379105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.379191 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.379489 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.379493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.379495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.379760 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:13.379983 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:13.380234 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 18:53:13.380412 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
W20260810 18:53:13.381902 31576 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40600
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[31344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[31344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:13.416893 32155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:13.407673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40602 (local address 127.30.153.129:36653)
0810 18:53:13.407987 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:13.407992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:13.408134 (+ 142us) server_negotiation.cc:407] Connection header received
0810 18:53:13.408324 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:13.408327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:13.408373 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:13.408465 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:13.409247 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.409762 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.410695 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:13.410923 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:13.411039 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:13.413723 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:13.413726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:13.413728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:13.413755 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:13.415291 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.415717 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.415720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.415721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.415796 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:13.416054 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:13.416056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:13.416057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:13.416247 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:13.416362 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:13.416633 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 18:53:13.416763 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260810 18:53:13.418298 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31512
I20260810 18:53:13.433308 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31717
I20260810 18:53:13.444651 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31936
I20260810 18:53:13.468931 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31364
2026-08-10T18:53:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2780 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[32167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[32167](info): set up 2 sockets
Aug 10 18:53:13 dist-test-slave-68xk krb5kdc[32167](info): commencing operation
krb5kdc: starting...
W20260810 18:53:14.824684 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:14 dist-test-slave-68xk krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:14.957455 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:39037
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:39037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:15.072540 32187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:15.072855 32187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:15.072924 32187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:15.076930 32187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:15.076987 32187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:15.077008 32187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:15.077025 32187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:15.077040 32187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:15.082867 32187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:39037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:39037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:15.083933 32187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:15.084914 32187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:15.094156 32196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.094285 32187 server_base.cc:1034] running on GCE node
W20260810 18:53:15.094225 32197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:15.094511 32199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.095183 32187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:15.096253 32187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:15.097460 32187 hybrid_clock.cc:648] HybridClock initialized: now 1786387995097448 us; error 29 us; skew 500 ppm
Aug 10 18:53:15 dist-test-slave-68xk krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387995, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:15.100495 32187 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:15.101465 32187 webserver.cc:460] Webserver started at http://127.30.153.190:34641/ using document root <none> and password file <none>
I20260810 18:53:15.102049 32187 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:15.102149 32187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:15.102401 32187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:15.104528 32187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "d3209f79f72b4debbda658c9be417d27"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "bb951ec00b5ee44a4dfa71a84d15f746"
server_key_iv: "af950d125e7bbc1040d6fe63ffe142c0"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.105140 32187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "d3209f79f72b4debbda658c9be417d27"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "bb951ec00b5ee44a4dfa71a84d15f746"
server_key_iv: "af950d125e7bbc1040d6fe63ffe142c0"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.109076 32187 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 18:53:15.111711 32210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.113024 32187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:53:15.113181 32187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "d3209f79f72b4debbda658c9be417d27"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "bb951ec00b5ee44a4dfa71a84d15f746"
server_key_iv: "af950d125e7bbc1040d6fe63ffe142c0"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.113298 32187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:15.137858 32187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:15.141100 32187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:15.141347 32187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:15.149456 32187 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:39037
I20260810 18:53:15.149487 32287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:39037 every 8 connection(s)
I20260810 18:53:15.150490 32187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:15.153455 32288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:15.154449 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 32187
I20260810 18:53:15.154578 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:15.154888 31334 external_mini_cluster.cc:1428] Setting key 91bf34ea2174ce6067d05b82673fdd6c
I20260810 18:53:15.160298 32288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27: Bootstrap starting.
I20260810 18:53:15.162923 32288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27: Neither blocks nor log segments found. Creating new log.
Aug 10 18:53:15 dist-test-slave-68xk krb5kdc[32167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:15.163746 32288 log.cc:824] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:15.166361 32288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27: No bootstrap required, opened a new log
I20260810 18:53:15.170104 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.156435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.30.153.190:39037)
0810 18:53:15.156891 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.156904 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.156939 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:53:15.157583 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.157600 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.157929 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.158265 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:15.159102 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.160066 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.161000 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.161465 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.161637 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.164356 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:15.164361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:15.164375 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:15.164402 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:15.166588 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.167031 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.167037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.167044 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.167203 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.167486 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.167490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.167492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.168038 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:15.168259 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.168472 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.168935 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":136,"threads_started":1}
I20260810 18:53:15.171808 32288 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3209f79f72b4debbda658c9be417d27" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39037 } }
I20260810 18:53:15.172040 32288 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:15.172127 32288 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3209f79f72b4debbda658c9be417d27, State: Initialized, Role: FOLLOWER
I20260810 18:53:15.172636 32288 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3209f79f72b4debbda658c9be417d27" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39037 } }
I20260810 18:53:15.172781 32288 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:15.172875 32288 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:15.172983 32288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:15.174007 32288 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3209f79f72b4debbda658c9be417d27" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39037 } }
I20260810 18:53:15.174397 32288 leader_election.cc:302] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3209f79f72b4debbda658c9be417d27; no voters:
I20260810 18:53:15.174866 32288 leader_election.cc:288] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:15.175040 32294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:15.175312 32294 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [term 1 LEADER]: Becoming Leader. State: Replica: d3209f79f72b4debbda658c9be417d27, State: Running, Role: LEADER
I20260810 18:53:15.175638 32294 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3209f79f72b4debbda658c9be417d27" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39037 } }
I20260810 18:53:15.176324 32288 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:15.177505 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3209f79f72b4debbda658c9be417d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3209f79f72b4debbda658c9be417d27" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39037 } } }
I20260810 18:53:15.177495 32298 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3209f79f72b4debbda658c9be417d27. Latest consensus state: current_term: 1 leader_uuid: "d3209f79f72b4debbda658c9be417d27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3209f79f72b4debbda658c9be417d27" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39037 } } }
I20260810 18:53:15.177671 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:15.177711 32298 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:15.178098 32307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:15.181162 32307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:15.186991 32307 catalog_manager.cc:1347] Generated new cluster ID: 5fae2ab1c70849bf921944c943289d74
I20260810 18:53:15.187072 32307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:15.200933 32307 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:15.201906 32307 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:15.206712 32307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3209f79f72b4debbda658c9be417d27: Generated new TSK 0
I20260810 18:53:15.207369 32307 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:15.241709 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39037
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:15.355993 32326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:15.356245 32326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:15.356289 32326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:15.360309 32326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:15.360361 32326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:15.360419 32326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:15.365490 32326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39037
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:15.366462 32326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:15.367502 32326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:15.377390 32338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:15.377502 32333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.377561 32326 server_base.cc:1034] running on GCE node
W20260810 18:53:15.377454 32334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.378162 32326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:15.379074 32326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:15.380285 32326 hybrid_clock.cc:648] HybridClock initialized: now 1786387995380250 us; error 47 us; skew 500 ppm
Aug 10 18:53:15 dist-test-slave-68xk krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387995, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:15.383354 32326 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:15.384335 32326 webserver.cc:460] Webserver started at http://127.30.153.129:33647/ using document root <none> and password file <none>
I20260810 18:53:15.384953 32326 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:15.385082 32326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:15.385349 32326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:15.387620 32326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "17aec7eb9cc7472292fc01d5e5e33e43"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "52d86903327216f66580a892faa3366a"
server_key_iv: "a6c522804883f77f15b8494cbd954ae9"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.388258 32326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "17aec7eb9cc7472292fc01d5e5e33e43"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "52d86903327216f66580a892faa3366a"
server_key_iv: "a6c522804883f77f15b8494cbd954ae9"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.392347 32326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:15.395275 32349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.396610 32326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:53:15.396762 32326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "17aec7eb9cc7472292fc01d5e5e33e43"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "52d86903327216f66580a892faa3366a"
server_key_iv: "a6c522804883f77f15b8494cbd954ae9"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.396884 32326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:15.424578 32326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:15.428139 32326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:15.428372 32326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:15.429020 32326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:15.429908 32326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:15.429984 32326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.430059 32326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:15.430104 32326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.439356 32326 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:33333
I20260810 18:53:15.439445 32547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:33333 every 8 connection(s)
I20260810 18:53:15.440304 32326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:15.448647 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 32326
I20260810 18:53:15.448776 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:15.449088 31334 external_mini_cluster.cc:1428] Setting key 78f2432918583cdc4faa82b8d0891c40
Aug 10 18:53:15 dist-test-slave-68xk krb5kdc[32167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387995, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:15.454728 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.442211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:40029 (local address 127.30.153.190:39037)
0810 18:53:15.442398 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.442402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.443248 (+ 846us) server_negotiation.cc:407] Connection header received
0810 18:53:15.444604 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.444607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.444668 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.444766 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:15.445939 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.446560 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.447718 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.447970 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.448080 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.450896 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:15.450899 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:15.450902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:15.450935 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:15.452381 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.452992 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.452998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.453000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.453101 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.453434 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.453440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.453443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.453712 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:15.453851 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.454312 (+ 461us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.454487 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 18:53:15.455463 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.442621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:39037 (local address 127.30.153.129:40029)
0810 18:53:15.443093 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:15.443138 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:15.444392 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:15.444942 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:15.444949 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:15.445324 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:15.445788 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.445802 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.446696 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.446700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.447577 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.447586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.448136 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.448139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.448318 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.448982 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:15.449002 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:15.450723 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:15.452544 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:15.452554 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:15.452573 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:15.452888 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:15.453198 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:15.453201 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:15.453202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:15.453337 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:15.453843 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:15.453848 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:15.454066 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 18:53:15.454221 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 18:53:15.454535 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":123,"threads_started":1}
I20260810 18:53:15.456775 32549 heartbeater.cc:347] Connected to a master server at 127.30.153.190:39037
I20260810 18:53:15.457027 32549 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:15.457569 32549 heartbeater.cc:511] Master 127.30.153.190:39037 requested a full tablet report, sending...
I20260810 18:53:15.459118 32234 ts_manager.cc:194] Registered new tserver with Master: 17aec7eb9cc7472292fc01d5e5e33e43 (127.30.153.129:33333)
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:53:15.460284 32234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:40029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:15.477840 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39037
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:15.590723 32559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:15.590996 32559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:15.591040 32559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:15.595206 32559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:15.595265 32559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:15.595324 32559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:15.600636 32559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39037
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:15.601749 32559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:15.602799 32559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:15.612656 32571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:15.612727 32570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.612807 32559 server_base.cc:1034] running on GCE node
W20260810 18:53:15.612653 32575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.613355 32559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:15.614001 32559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:15.615188 32559 hybrid_clock.cc:648] HybridClock initialized: now 1786387995615157 us; error 45 us; skew 500 ppm
Aug 10 18:53:15 dist-test-slave-68xk krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387995, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:15.618147 32559 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:15.619061 32559 webserver.cc:460] Webserver started at http://127.30.153.130:38201/ using document root <none> and password file <none>
I20260810 18:53:15.619640 32559 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:15.619738 32559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:15.619979 32559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:15.622040 32559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "3606a9f38162406bae7afbb8bba9a0e7"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "e510137cb5766e3f3ab67809a5c9bf16"
server_key_iv: "76a76c477bdf886d3580580a219c9fed"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.622671 32559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "3606a9f38162406bae7afbb8bba9a0e7"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "e510137cb5766e3f3ab67809a5c9bf16"
server_key_iv: "76a76c477bdf886d3580580a219c9fed"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.626475 32559 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:15.628970 32585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.630250 32559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:15.630389 32559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "3606a9f38162406bae7afbb8bba9a0e7"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "e510137cb5766e3f3ab67809a5c9bf16"
server_key_iv: "76a76c477bdf886d3580580a219c9fed"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.630573 32559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:15.651450 32559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:15.654973 32559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:15.655200 32559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:15.655846 32559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:15.656735 32559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:15.656809 32559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.656885 32559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:15.656939 32559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.666719 32559 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:36573
I20260810 18:53:15.666749 32765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:36573 every 8 connection(s)
I20260810 18:53:15.667613 32559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:15.674726 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 32559
I20260810 18:53:15.674837 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:15.675117 31334 external_mini_cluster.cc:1428] Setting key cf3a39569f5c4415109c52238fe3953c
Aug 10 18:53:15 dist-test-slave-68xk krb5kdc[32167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387995, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:15.683430 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.669533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:43957 (local address 127.30.153.190:39037)
0810 18:53:15.669689 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.669693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.670605 (+ 912us) server_negotiation.cc:407] Connection header received
0810 18:53:15.671969 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.671972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.672024 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.672112 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:15.673289 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.673834 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.675366 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.675613 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.675738 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.679053 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:15.679059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:15.679064 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:15.679100 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:15.680856 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.681429 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.681435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.681440 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.681543 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.682003 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.682010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.682015 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.682301 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:15.682449 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.683021 (+ 572us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.683166 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:53:15.684106 305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.669889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:39037 (local address 127.30.153.130:43957)
0810 18:53:15.670373 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:15.670482 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:15.671749 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:15.672270 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:15.672277 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:15.672633 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:15.673125 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.673138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.673959 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.673962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.675196 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.675209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.675796 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.675800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.676025 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.676725 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:15.676749 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:15.678807 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:15.680990 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:15.680995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:15.681008 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:15.681290 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:15.681653 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:15.681665 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:15.681667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:15.681872 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:15.682461 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:15.682467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:15.682741 (+ 274us) client_negotiation.cc:769] Sending connection context
0810 18:53:15.682906 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 18:53:15.683180 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":123,"threads_started":1}
I20260810 18:53:15.685418 32766 heartbeater.cc:347] Connected to a master server at 127.30.153.190:39037
I20260810 18:53:15.685691 32766 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:15.686188 32766 heartbeater.cc:511] Master 127.30.153.190:39037 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:53:15.687397 32234 ts_manager.cc:194] Registered new tserver with Master: 3606a9f38162406bae7afbb8bba9a0e7 (127.30.153.130:36573)
I20260810 18:53:15.688205 32234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:43957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:15.709599 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39037
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:15.825791 310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:15.826081 310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:15.826179 310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:15.830257 310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:15.830345 310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:15.830502 310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:15.835861 310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39037
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:15.836867 310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:15.837855 310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:15.847815 322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.847786 310 server_base.cc:1034] running on GCE node
W20260810 18:53:15.847779 318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:15.847942 317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:15.848481 310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:15.849128 310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:15.850339 310 hybrid_clock.cc:648] HybridClock initialized: now 1786387995850317 us; error 33 us; skew 500 ppm
Aug 10 18:53:15 dist-test-slave-68xk krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387995, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:15.853420 310 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:15.854374 310 webserver.cc:460] Webserver started at http://127.30.153.131:43173/ using document root <none> and password file <none>
I20260810 18:53:15.855000 310 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:15.855096 310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:15.855337 310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:15.857419 310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "2d217fe4bce44aa4b8448647e1427612"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "147b4c59fba144b0b076568a0a37c4fc"
server_key_iv: "e7d8ff0d99e007868662daa58757dcb2"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.857999 310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "2d217fe4bce44aa4b8448647e1427612"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "147b4c59fba144b0b076568a0a37c4fc"
server_key_iv: "e7d8ff0d99e007868662daa58757dcb2"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.861833 310 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260810 18:53:15.864331 334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.865523 310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:53:15.865695 310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "2d217fe4bce44aa4b8448647e1427612"
format_stamp: "Formatted at 2026-08-10 18:53:15 on dist-test-slave-68xk"
server_key: "147b4c59fba144b0b076568a0a37c4fc"
server_key_iv: "e7d8ff0d99e007868662daa58757dcb2"
server_key_version: "encryptionkey@0"
I20260810 18:53:15.865814 310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:15.876302 310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:15.879571 310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:15.879786 310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:15.880373 310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:15.881199 310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:15.881271 310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.881336 310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:15.881386 310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:15.891399 310 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:37641
I20260810 18:53:15.891475 513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:37641 every 8 connection(s)
I20260810 18:53:15.892304 310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:15.897511 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 310
I20260810 18:53:15.897609 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:15.897887 31334 external_mini_cluster.cc:1428] Setting key 3e516673d18b6e9a9a5c7ca0201deed6
Aug 10 18:53:15 dist-test-slave-68xk krb5kdc[32167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387995, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:15.907758 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.894208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:50511 (local address 127.30.153.190:39037)
0810 18:53:15.894354 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.894358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.895338 (+ 980us) server_negotiation.cc:407] Connection header received
0810 18:53:15.897030 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.897034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.897109 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.897226 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:15.898763 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.899357 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.900749 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.901032 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.901167 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.903864 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:15.903869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:15.903871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:15.903901 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:15.905435 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.906021 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.906025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.906027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.906134 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.906567 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.906570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.906572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.906786 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:15.906908 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.907440 (+ 532us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.907574 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:15.908595 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.894635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:39037 (local address 127.30.153.131:50511)
0810 18:53:15.895071 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:15.895127 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:15.896790 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:15.897409 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:15.897417 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:15.897910 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:15.898604 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.898623 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.899507 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.899511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.900611 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.900622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.901219 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.901224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.901406 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.902087 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:15.902106 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:15.903682 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:15.905593 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:15.905599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:15.905614 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:15.905894 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:15.906244 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:15.906248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:15.906251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:15.906403 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:15.906909 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:15.906917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:15.907177 (+ 260us) client_negotiation.cc:769] Sending connection context
0810 18:53:15.907376 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 18:53:15.907729 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":109,"threads_started":1}
I20260810 18:53:15.909783 514 heartbeater.cc:347] Connected to a master server at 127.30.153.190:39037
I20260810 18:53:15.910007 514 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:15.910596 514 heartbeater.cc:511] Master 127.30.153.190:39037 requested a full tablet report, sending...
I20260810 18:53:15.912075 32233 ts_manager.cc:194] Registered new tserver with Master: 2d217fe4bce44aa4b8448647e1427612 (127.30.153.131:37641)
I20260810 18:53:15.912866 32233 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:50511
I20260810 18:53:15.924213 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:15.937692 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.928901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.30.153.190:39037)
0810 18:53:15.929066 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.929071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.929316 (+ 245us) server_negotiation.cc:407] Connection header received
0810 18:53:15.929426 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.929428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.929474 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.929547 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:15.930573 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.931101 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.932048 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.932309 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.932397 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.934247 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:15.934251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:15.934253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:15.934280 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:15.935969 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.936416 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.936419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.936420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.936504 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:15.936769 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:15.936771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:15.936772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:15.936999 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:15.937104 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.937341 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.937446 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:53:15.944550 32233 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:15.948316 32233 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:15.965337 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.960249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.30.153.131:37641)
0810 18:53:15.960729 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.960743 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.960766 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:53:15.960847 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.960850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.961046 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.961230 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:15.961439 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.962551 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.964033 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.964610 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.964767 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.964957 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.965084 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.965177 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":165,"threads_started":1}
I20260810 18:53:15.965489 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.959032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:33333 (local address 127.0.0.1:54360)
0810 18:53:15.959609 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:15.959670 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:15.959794 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:15.960650 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:15.960656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:15.960681 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:15.960863 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.960871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.962155 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.962158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.963756 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.963767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.964972 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.964976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.965095 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.965144 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 18:53:15.965214 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 18:53:15.965303 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":19,"thread_start_us":108,"threads_started":1}
I20260810 18:53:15.965327 533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.959810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:37641 (local address 127.0.0.1:51890)
0810 18:53:15.960582 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:15.960599 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:15.960708 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:15.961223 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:15.961228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:15.961249 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:15.961337 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.961343 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.962813 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.962816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.963901 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.963912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.964744 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.964748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.964870 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.964912 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 18:53:15.965055 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 18:53:15.965190 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":83,"thread_start_us":94,"threads_started":1}
I20260810 18:53:15.966001 534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.959525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:36573 (local address 127.0.0.1:38984)
0810 18:53:15.960347 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:15.960360 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:15.960441 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:15.960985 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:15.960988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:15.961004 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:15.961135 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.961145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.962703 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.962707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.963961 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:15.963993 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.965523 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.965527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:15.965683 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.965707 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:53:15.965781 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 18:53:15.965853 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":93,"thread_start_us":103,"threads_started":1}
I20260810 18:53:15.966246 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.959221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54360 (local address 127.30.153.129:33333)
0810 18:53:15.960108 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.960116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.960141 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:53:15.960240 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.960247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.960467 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.961016 (+ 549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:15.961181 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.962034 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.963901 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.964787 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.965418 (+ 631us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.965531 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.965579 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.966081 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":131,"threads_started":1}
I20260810 18:53:15.966526 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:15.959963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38984 (local address 127.30.153.130:36573)
0810 18:53:15.960394 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:15.960409 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:15.960438 (+ 29us) server_negotiation.cc:407] Connection header received
0810 18:53:15.960615 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:15.960621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:15.960831 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:15.960999 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:15.961280 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.962532 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.964085 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:15.965219 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:15.965333 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:15.965458 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:15.966237 (+ 779us) server_negotiation.cc:299] Negotiation successful
0810 18:53:15.966352 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":112,"threads_started":1}
I20260810 18:53:15.968966 32656 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd33db7899245cf88e2a65e0488af77 (DEFAULT_TABLE table=test-table [id=8f22d7b656874d848a3d55b75b2c31e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:15.968986 403 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd33db7899245cf88e2a65e0488af77 (DEFAULT_TABLE table=test-table [id=8f22d7b656874d848a3d55b75b2c31e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:15.968991 32443 tablet_service.cc:1467] Processing CreateTablet for tablet 1dd33db7899245cf88e2a65e0488af77 (DEFAULT_TABLE table=test-table [id=8f22d7b656874d848a3d55b75b2c31e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:15.970109 403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd33db7899245cf88e2a65e0488af77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:15.972011 32656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd33db7899245cf88e2a65e0488af77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:15.972707 32443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dd33db7899245cf88e2a65e0488af77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:15.978660 540 tablet_bootstrap.cc:492] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612: Bootstrap starting.
I20260810 18:53:15.979912 543 tablet_bootstrap.cc:492] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7: Bootstrap starting.
I20260810 18:53:15.980324 547 tablet_bootstrap.cc:492] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43: Bootstrap starting.
I20260810 18:53:15.983024 547 tablet_bootstrap.cc:654] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:15.983125 540 tablet_bootstrap.cc:654] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:15.983831 543 tablet_bootstrap.cc:654] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:15.984441 547 log.cc:824] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:15.985431 543 log.cc:824] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:15.985432 540 log.cc:824] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:15.986953 547 tablet_bootstrap.cc:492] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43: No bootstrap required, opened a new log
I20260810 18:53:15.987174 547 ts_tablet_manager.cc:1397] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:53:15.987617 540 tablet_bootstrap.cc:492] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612: No bootstrap required, opened a new log
I20260810 18:53:15.987812 540 ts_tablet_manager.cc:1397] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 18:53:15.988031 543 tablet_bootstrap.cc:492] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7: No bootstrap required, opened a new log
I20260810 18:53:15.988221 543 ts_tablet_manager.cc:1397] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 18:53:15.992065 540 raft_consensus.cc:348] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:15.992372 540 raft_consensus.cc:374] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:15.992429 540 raft_consensus.cc:729] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d217fe4bce44aa4b8448647e1427612, State: Initialized, Role: FOLLOWER
I20260810 18:53:15.992878 540 consensus_queue.cc:260] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:15.993717 514 heartbeater.cc:503] Master 127.30.153.190:39037 was elected leader, sending a full tablet report...
I20260810 18:53:15.994057 540 ts_tablet_manager.cc:1428] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:53:15.994678 547 raft_consensus.cc:348] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:15.994678 543 raft_consensus.cc:348] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:15.994971 547 raft_consensus.cc:374] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:15.994971 543 raft_consensus.cc:374] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:15.995033 547 raft_consensus.cc:729] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17aec7eb9cc7472292fc01d5e5e33e43, State: Initialized, Role: FOLLOWER
I20260810 18:53:15.995033 543 raft_consensus.cc:729] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3606a9f38162406bae7afbb8bba9a0e7, State: Initialized, Role: FOLLOWER
I20260810 18:53:15.995484 547 consensus_queue.cc:260] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:15.995468 543 consensus_queue.cc:260] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:15.996603 32549 heartbeater.cc:503] Master 127.30.153.190:39037 was elected leader, sending a full tablet report...
I20260810 18:53:15.996639 547 ts_tablet_manager.cc:1428] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260810 18:53:15.997381 32766 heartbeater.cc:503] Master 127.30.153.190:39037 was elected leader, sending a full tablet report...
I20260810 18:53:15.997684 543 ts_tablet_manager.cc:1428] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7: Time spent starting tablet: real 0.009s user 0.005s sys 0.004s
W20260810 18:53:16.143488 516 tablet.cc:2369] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:16.168967 304 tablet.cc:2369] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:16.191771 32550 tablet.cc:2369] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:16.212563 553 raft_consensus.cc:482] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:16.212823 553 raft_consensus.cc:504] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:16.214008 553 leader_election.cc:288] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17aec7eb9cc7472292fc01d5e5e33e43 (127.30.153.129:33333), 2d217fe4bce44aa4b8448647e1427612 (127.30.153.131:37641)
I20260810 18:53:16.218720 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.214685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:42229 (local address 127.30.153.129:33333)
0810 18:53:16.214849 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.214854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.214876 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:53:16.214968 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.214971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.215043 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.215125 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:16.215650 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.216149 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.217524 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.218032 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.218178 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.218234 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.218518 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.218597 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:16.218816 555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.214361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:33333 (local address 127.30.153.130:42229)
0810 18:53:16.214730 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:16.214747 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:16.214853 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:16.215161 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:16.215166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:16.215319 (+ 153us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:16.215509 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.215515 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.216324 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.216327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.217355 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.217371 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.218173 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.218177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.218335 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.218357 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:53:16.218493 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:53:16.218570 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":121,"threads_started":1}
I20260810 18:53:16.219028 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.215136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:46571 (local address 127.30.153.131:37641)
0810 18:53:16.215267 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.215271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.215284 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:16.215429 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.215433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.215482 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.215566 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:16.215934 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.216533 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.217873 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.218381 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.218553 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.218613 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.218850 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.218920 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 18:53:16.219105 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.214885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:37641 (local address 127.30.153.130:46571)
0810 18:53:16.215154 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:16.215169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:16.215308 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:16.215578 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:16.215581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:16.215597 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:16.215810 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.215816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.216649 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.216656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.217749 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:16.217760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.218547 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.218551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:16.218715 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.218735 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:53:16.218831 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:53:16.218902 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":69,"threads_started":1}
I20260810 18:53:16.219660 32473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd33db7899245cf88e2a65e0488af77" candidate_uuid: "3606a9f38162406bae7afbb8bba9a0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" is_pre_election: true
I20260810 18:53:16.219738 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd33db7899245cf88e2a65e0488af77" candidate_uuid: "3606a9f38162406bae7afbb8bba9a0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d217fe4bce44aa4b8448647e1427612" is_pre_election: true
I20260810 18:53:16.220129 32473 raft_consensus.cc:2454] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3606a9f38162406bae7afbb8bba9a0e7 in term 0.
I20260810 18:53:16.220155 437 raft_consensus.cc:2454] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3606a9f38162406bae7afbb8bba9a0e7 in term 0.
I20260810 18:53:16.220793 32589 leader_election.cc:302] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17aec7eb9cc7472292fc01d5e5e33e43, 3606a9f38162406bae7afbb8bba9a0e7; no voters:
I20260810 18:53:16.221056 553 raft_consensus.cc:2781] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:16.221150 553 raft_consensus.cc:482] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:16.221196 553 raft_consensus.cc:3037] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:16.222688 553 raft_consensus.cc:504] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:16.223021 553 leader_election.cc:288] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [CANDIDATE]: Term 1 election: Requested vote from peers 17aec7eb9cc7472292fc01d5e5e33e43 (127.30.153.129:33333), 2d217fe4bce44aa4b8448647e1427612 (127.30.153.131:37641)
I20260810 18:53:16.223502 32473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd33db7899245cf88e2a65e0488af77" candidate_uuid: "3606a9f38162406bae7afbb8bba9a0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17aec7eb9cc7472292fc01d5e5e33e43"
I20260810 18:53:16.223546 437 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dd33db7899245cf88e2a65e0488af77" candidate_uuid: "3606a9f38162406bae7afbb8bba9a0e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d217fe4bce44aa4b8448647e1427612"
I20260810 18:53:16.223666 32473 raft_consensus.cc:3037] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:16.223675 437 raft_consensus.cc:3037] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:16.225292 437 raft_consensus.cc:2454] T 1dd33db7899245cf88e2a65e0488af77 P 2d217fe4bce44aa4b8448647e1427612 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3606a9f38162406bae7afbb8bba9a0e7 in term 1.
I20260810 18:53:16.225695 32589 leader_election.cc:302] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d217fe4bce44aa4b8448647e1427612, 3606a9f38162406bae7afbb8bba9a0e7; no voters:
I20260810 18:53:16.225782 32473 raft_consensus.cc:2454] T 1dd33db7899245cf88e2a65e0488af77 P 17aec7eb9cc7472292fc01d5e5e33e43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3606a9f38162406bae7afbb8bba9a0e7 in term 1.
I20260810 18:53:16.225864 553 raft_consensus.cc:2781] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:16.226117 553 raft_consensus.cc:686] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [term 1 LEADER]: Becoming Leader. State: Replica: 3606a9f38162406bae7afbb8bba9a0e7, State: Running, Role: LEADER
I20260810 18:53:16.226536 553 consensus_queue.cc:237] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } }
I20260810 18:53:16.230026 32234 catalog_manager.cc:5495] T 1dd33db7899245cf88e2a65e0488af77 P 3606a9f38162406bae7afbb8bba9a0e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3606a9f38162406bae7afbb8bba9a0e7 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "3606a9f38162406bae7afbb8bba9a0e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17aec7eb9cc7472292fc01d5e5e33e43" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d217fe4bce44aa4b8448647e1427612" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3606a9f38162406bae7afbb8bba9a0e7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36573 } health_report { overall_health: HEALTHY } } }
Aug 10 18:53:16 dist-test-slave-68xk krb5kdc[32167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:16.259675 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.249794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.30.153.129:33333)
0810 18:53:16.250007 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.250013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.250343 (+ 330us) server_negotiation.cc:407] Connection header received
0810 18:53:16.250490 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.250493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.250537 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.250613 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:16.251401 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.251919 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.252964 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.253174 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.253258 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.255866 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:16.255884 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:16.255899 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:16.255924 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:16.257950 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:16.258370 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:16.258375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:16.258376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:16.258512 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:16.258761 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:16.258763 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:16.258764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:16.258995 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:16.259144 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.259354 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.259484 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 18:53:16 dist-test-slave-68xk krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387996, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:53:16 dist-test-slave-68xk krb5kdc[32167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387996, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:16.290884 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.281952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.30.153.129:33333)
0810 18:53:16.282134 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.282138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.282200 (+ 62us) server_negotiation.cc:407] Connection header received
0810 18:53:16.282328 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.282331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.282375 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.282513 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:16.283177 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.283820 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.284725 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.285068 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.285181 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.287820 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:16.287823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:16.287825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:16.287850 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:16.289218 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:16.289628 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:16.289630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:16.289632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:16.289709 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:16.289976 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:16.289978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:16.289980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:16.290139 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:16.290241 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.290587 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.290698 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 10 18:53:16 dist-test-slave-68xk krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:53:16 dist-test-slave-68xk krb5kdc[32167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:16.321190 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:16.311503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.30.153.129:33333)
0810 18:53:16.311679 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:16.311684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:16.311701 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:53:16.311834 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:16.311837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:16.311880 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:16.311953 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:16.312771 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.313271 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.314235 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:16.314506 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:16.314592 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:16.317731 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:16.317736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:16.317738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:16.317771 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:16.319342 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:16.319798 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:16.319800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:16.319801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:16.319877 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:16.320170 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:16.320174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:16.320175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:16.320375 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:16.320501 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:16.320817 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 18:53:16.320976 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 18:53:16.323026 32409 tablet_service.cc:2908] Rejecting scan request for tablet 1dd33db7899245cf88e2a65e0488af77: Uninitialized: clean time has not yet been initialized
I20260810 18:53:16.325623 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 32326
I20260810 18:53:16.343257 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 32559
I20260810 18:53:16.357237 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 310
I20260810 18:53:16.379765 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 32187
2026-08-10T18:53:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2908 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:16 dist-test-slave-68xk krb5kdc[584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:16 dist-test-slave-68xk krb5kdc[584](info): set up 2 sockets
Aug 10 18:53:16 dist-test-slave-68xk krb5kdc[584](info): commencing operation
krb5kdc: starting...
W20260810 18:53:17.638717 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:17 dist-test-slave-68xk krb5kdc[584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:17.773536 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42897
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:42897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:17.887936 612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:17.888239 612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:17.888321 612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:17.892412 612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:17.892495 612 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:53:17.892529 612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:17.892570 612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:17.892607 612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:17.892642 612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:17.897853 612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:42897
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:17.899029 612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:17.900100 612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:17.908793 625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:17.908905 612 server_base.cc:1034] running on GCE node
W20260810 18:53:17.908828 621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:17.908836 622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:17.909667 612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:17.910928 612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:17.912153 612 hybrid_clock.cc:648] HybridClock initialized: now 1786387997912124 us; error 47 us; skew 500 ppm
Aug 10 18:53:17 dist-test-slave-68xk krb5kdc[584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387997, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:17.915208 612 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:17.916101 612 webserver.cc:460] Webserver started at http://127.30.153.190:44763/ using document root <none> and password file <none>
I20260810 18:53:17.916646 612 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:17.916710 612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:17.916875 612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:17.919015 612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "eb9270336f184ffc8b6cbf8e0d818158"
format_stamp: "Formatted at 2026-08-10 18:53:17 on dist-test-slave-68xk"
server_key: "596c997fd59de2bc89af88ad2b233f6c"
server_key_iv: "978fa4a50148acd7f8e7198072eeb3ef"
server_key_version: "encryptionkey@0"
I20260810 18:53:17.919575 612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "eb9270336f184ffc8b6cbf8e0d818158"
format_stamp: "Formatted at 2026-08-10 18:53:17 on dist-test-slave-68xk"
server_key: "596c997fd59de2bc89af88ad2b233f6c"
server_key_iv: "978fa4a50148acd7f8e7198072eeb3ef"
server_key_version: "encryptionkey@0"
I20260810 18:53:17.923460 612 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:17.925997 635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:17.927420 612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:17.927556 612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "eb9270336f184ffc8b6cbf8e0d818158"
format_stamp: "Formatted at 2026-08-10 18:53:17 on dist-test-slave-68xk"
server_key: "596c997fd59de2bc89af88ad2b233f6c"
server_key_iv: "978fa4a50148acd7f8e7198072eeb3ef"
server_key_version: "encryptionkey@0"
I20260810 18:53:17.927670 612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:17.939314 612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:17.942691 612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:17.942894 612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:17.951749 612 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:42897
I20260810 18:53:17.951824 750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:42897 every 8 connection(s)
I20260810 18:53:17.952800 612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:17.955859 752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:17.960034 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 612
I20260810 18:53:17.960192 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:17.960458 31334 external_mini_cluster.cc:1428] Setting key 7346b355ffb7c896a385a28701091546
I20260810 18:53:17.962224 752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158: Bootstrap starting.
I20260810 18:53:17.964931 752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:17.965939 752 log.cc:824] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:17.968261 752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158: No bootstrap required, opened a new log
Aug 10 18:53:17 dist-test-slave-68xk krb5kdc[584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:17.972812 752 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9270336f184ffc8b6cbf8e0d818158" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42897 } }
I20260810 18:53:17.973111 752 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:17.973178 752 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb9270336f184ffc8b6cbf8e0d818158, State: Initialized, Role: FOLLOWER
I20260810 18:53:17.973695 752 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9270336f184ffc8b6cbf8e0d818158" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42897 } }
I20260810 18:53:17.973835 752 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:17.973932 752 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:17.974038 752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:17.975267 752 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9270336f184ffc8b6cbf8e0d818158" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42897 } }
I20260810 18:53:17.975663 752 leader_election.cc:302] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb9270336f184ffc8b6cbf8e0d818158; no voters:
I20260810 18:53:17.976042 752 leader_election.cc:288] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:17.976205 759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:17.976514 759 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [term 1 LEADER]: Becoming Leader. State: Replica: eb9270336f184ffc8b6cbf8e0d818158, State: Running, Role: LEADER
I20260810 18:53:17.976859 759 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9270336f184ffc8b6cbf8e0d818158" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42897 } }
I20260810 18:53:17.977085 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:17.962071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53156 (local address 127.30.153.190:42897)
0810 18:53:17.962568 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:17.962586 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:17.962626 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:53:17.963377 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:17.963396 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:17.963752 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:17.964088 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:17.965138 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.966674 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.967582 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:17.968273 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:17.968441 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:17.971304 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:17.971311 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:17.971327 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:17.971361 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:17.973484 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:17.974102 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:17.974109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:17.974116 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:17.974284 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:17.974641 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:17.974645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:17.974647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:17.975203 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:17.975442 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:17.975757 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 18:53:17.976082 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":164,"threads_started":1}
I20260810 18:53:17.977375 752 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:17.978833 761 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb9270336f184ffc8b6cbf8e0d818158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9270336f184ffc8b6cbf8e0d818158" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42897 } } }
I20260810 18:53:17.978953 761 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:17.978895 762 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9270336f184ffc8b6cbf8e0d818158. Latest consensus state: current_term: 1 leader_uuid: "eb9270336f184ffc8b6cbf8e0d818158" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9270336f184ffc8b6cbf8e0d818158" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42897 } } }
I20260810 18:53:17.979007 762 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:17.979496 767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:17.982785 767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:17.988456 767 catalog_manager.cc:1347] Generated new cluster ID: e5010c8a79464d62a23f5c9c08cfe60b
I20260810 18:53:17.988533 767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:17.996893 767 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:17.998154 767 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:18.004853 767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158: Generated new TSK 0
I20260810 18:53:18.005568 767 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:18.038370 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42897
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:18.158624 787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:18.158872 787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:18.158917 787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:18.162971 787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:18.163030 787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:18.163075 787 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:18.163170 787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:18.168459 787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:42897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:18.169557 787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:18.170636 787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:18.180526 794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:18.180588 787 server_base.cc:1034] running on GCE node
W20260810 18:53:18.180532 793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:18.180532 796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:18.181097 787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:18.181679 787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:18.182857 787 hybrid_clock.cc:648] HybridClock initialized: now 1786387998182831 us; error 39 us; skew 500 ppm
Aug 10 18:53:18 dist-test-slave-68xk krb5kdc[584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:18.189216 787 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:18.190138 787 webserver.cc:460] Webserver started at http://127.30.153.129:39257/ using document root <none> and password file <none>
I20260810 18:53:18.190773 787 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:18.190840 787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:18.191010 787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:18.193010 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "8fbadda1fe7e4eb5b877bc3e803f9182"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "3389b56ba9c7c6b97da83bdafdb52da9"
server_key_iv: "90747f713fb05b0bb5283c1fce65ac24"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.193523 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "8fbadda1fe7e4eb5b877bc3e803f9182"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "3389b56ba9c7c6b97da83bdafdb52da9"
server_key_iv: "90747f713fb05b0bb5283c1fce65ac24"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.197466 787 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:53:18.200060 803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:18.201465 787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:18.201622 787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "8fbadda1fe7e4eb5b877bc3e803f9182"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "3389b56ba9c7c6b97da83bdafdb52da9"
server_key_iv: "90747f713fb05b0bb5283c1fce65ac24"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.201740 787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:18.227300 787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:18.230994 787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:18.231256 787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:18.233062 787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:18.233150 787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:18.233215 787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:18.233266 787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:18 dist-test-slave-68xk krb5kdc[584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:18.244282 787 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:41509
I20260810 18:53:18.244798 982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:41509 every 8 connection(s)
I20260810 18:53:18.245404 787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:18.246069 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 787
I20260810 18:53:18.246181 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:18.246543 31334 external_mini_cluster.cc:1428] Setting key 19a39f4183edec93578211f0d79f0783
I20260810 18:53:18.248497 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.233858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:47715 (local address 127.30.153.190:42897)
0810 18:53:18.234026 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.234032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.235522 (+ 1490us) server_negotiation.cc:407] Connection header received
0810 18:53:18.237017 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.237023 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.237086 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.237191 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:18.239014 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.239791 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.240957 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.241209 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.241322 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.244318 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:18.244321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:18.244324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:18.244352 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:18.246199 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.246877 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.246880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.246882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.246979 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.247366 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.247370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.247372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.247575 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:18.247689 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.248190 (+ 501us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.248323 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:53:18.249285 816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.234803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42897 (local address 127.30.153.129:47715)
0810 18:53:18.235356 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.235414 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.236754 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.237417 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.237427 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.237981 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:18.238829 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.238854 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.239913 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.239921 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.240824 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.240834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.241377 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.241381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.241550 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.242156 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:18.242179 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:18.244145 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:18.246342 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.246348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.246362 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.246752 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.247098 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.247102 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.247104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.247255 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.247688 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:18.247695 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:18.247944 (+ 249us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.248132 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.248469 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":150,"threads_started":1}
I20260810 18:53:18.250833 983 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42897
I20260810 18:53:18.251010 983 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:18.251683 983 heartbeater.cc:511] Master 127.30.153.190:42897 requested a full tablet report, sending...
I20260810 18:53:18.253378 661 ts_manager.cc:194] Registered new tserver with Master: 8fbadda1fe7e4eb5b877bc3e803f9182 (127.30.153.129:41509)
I20260810 18:53:18.254860 661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:47715
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
I20260810 18:53:18.261837 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.253325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42897 (local address 127.30.153.129:44575)
0810 18:53:18.253661 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.253677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.253793 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.254055 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.254060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.254311 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:18.254498 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.254510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.255460 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.255465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.256485 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.256497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.257079 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.257082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.257205 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.257729 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:18.257748 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:18.258239 (+ 491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:18.260254 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.260257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.260258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.260484 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.260753 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.260755 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.260756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.260837 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.261340 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:18.261342 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:18.261421 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.261509 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.261615 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":115,"threads_started":1}
I20260810 18:53:18.262051 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.253119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:44575 (local address 127.30.153.190:42897)
0810 18:53:18.253235 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.253239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.253770 (+ 531us) server_negotiation.cc:407] Connection header received
0810 18:53:18.253887 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.253890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.253931 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.254013 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:18.254628 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.255311 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.256625 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.256937 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.257059 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.258387 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:18.258390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:18.258391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:18.258461 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:18.260141 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.260584 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.260587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.260588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.260665 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.260978 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.260982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.260984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.261241 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:18.261324 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.261713 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.261831 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:18.278644 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42897
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:18.391078 997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:18.391356 997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:18.391433 997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:18.395391 997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:18.395467 997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:18.395533 997 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:18.395612 997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:18.401357 997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:42897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:18.402372 997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:18.403432 997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:18.413563 1007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:18.413635 1009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:18.413563 1004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:18.414127 997 server_base.cc:1034] running on GCE node
I20260810 18:53:18.414578 997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:18.415267 997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:18.416435 997 hybrid_clock.cc:648] HybridClock initialized: now 1786387998416414 us; error 30 us; skew 500 ppm
Aug 10 18:53:18 dist-test-slave-68xk krb5kdc[584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:18.419241 997 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:18.420109 997 webserver.cc:460] Webserver started at http://127.30.153.130:40961/ using document root <none> and password file <none>
I20260810 18:53:18.420681 997 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:18.420746 997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:18.421031 997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:18.423187 997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "dd573e73fcb74534a223e521a1d4c068"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "c720cbc011811cfc3deb2e9d5defd3d5"
server_key_iv: "94c9c87705ce0b1c11738c7755aa2fd3"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.423766 997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "dd573e73fcb74534a223e521a1d4c068"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "c720cbc011811cfc3deb2e9d5defd3d5"
server_key_iv: "94c9c87705ce0b1c11738c7755aa2fd3"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.427620 997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260810 18:53:18.430023 1019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:18.431354 997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:18.431500 997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "dd573e73fcb74534a223e521a1d4c068"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "c720cbc011811cfc3deb2e9d5defd3d5"
server_key_iv: "94c9c87705ce0b1c11738c7755aa2fd3"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.431613 997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:18.454074 997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:18.457494 997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:18.457751 997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:18.459484 997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:18.459539 997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:18.459612 997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:18.459652 997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:18 dist-test-slave-68xk krb5kdc[584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:18.472393 997 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:38025
I20260810 18:53:18.472462 1203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:38025 every 8 connection(s)
I20260810 18:53:18.473465 997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:18.475404 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.460541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:39999 (local address 127.30.153.190:42897)
0810 18:53:18.460686 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.460690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.461822 (+ 1132us) server_negotiation.cc:407] Connection header received
0810 18:53:18.463018 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.463022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.463086 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.463153 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:18.464967 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.465992 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.467580 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.468069 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.468225 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.471075 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:18.471078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:18.471080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:18.471106 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:18.472785 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.473422 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.473425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.473426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.473502 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.474007 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.474010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.474011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.474245 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:18.474350 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.475081 (+ 731us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.475219 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:53:18.476106 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 997
I20260810 18:53:18.476226 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:18.476398 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.461137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42897 (local address 127.30.153.130:39999)
0810 18:53:18.461657 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.461700 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.462749 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.463627 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.463638 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.464113 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:18.464772 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.464795 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.466145 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.466149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.467436 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.467449 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.468281 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.468287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.468498 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.469145 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:18.469162 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:18.470897 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:18.472908 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.472913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.472928 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.473311 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.473677 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.473681 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.473683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.473877 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.474370 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:18.474376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:18.474781 (+ 405us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.474982 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.475248 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":139,"threads_started":1}
I20260810 18:53:18.476547 31334 external_mini_cluster.cc:1428] Setting key ed0ae1ea3bab36d617c104b777c5f9ff
I20260810 18:53:18.477923 1206 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42897
I20260810 18:53:18.478091 1206 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:18.478739 1206 heartbeater.cc:511] Master 127.30.153.190:42897 requested a full tablet report, sending...
I20260810 18:53:18.479760 660 ts_manager.cc:194] Registered new tserver with Master: dd573e73fcb74534a223e521a1d4c068 (127.30.153.130:38025)
I20260810 18:53:18.480533 660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:39999
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:53:18.488817 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.480370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:40759 (local address 127.30.153.190:42897)
0810 18:53:18.480461 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.480465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.481024 (+ 559us) server_negotiation.cc:407] Connection header received
0810 18:53:18.481142 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.481146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.481187 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.481261 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:18.481895 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.482636 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.483714 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.483969 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.484091 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.485264 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:18.485267 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:18.485270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:18.485303 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:18.486965 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.487429 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.487434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.487436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.487528 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.487848 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.487853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.487855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.488070 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:18.488222 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.488523 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.488651 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 18:53:18.488826 1214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.480561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42897 (local address 127.30.153.130:40759)
0810 18:53:18.480916 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.480934 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.481046 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.481314 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.481320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.481573 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:18.481739 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.481750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.482764 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.482766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.483576 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.483587 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.484097 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.484101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.484206 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.484743 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:18.484755 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:18.485090 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:18.487088 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.487093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.487095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.487319 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.487643 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.487646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.487650 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.487745 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.488222 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:18.488226 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:18.488332 (+ 106us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.488464 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.488626 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":155,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:18.509482 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42897
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:18.624009 1221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:18.624290 1221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:18.624337 1221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:18.628589 1221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:18.628655 1221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:18.628695 1221 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:18.628729 1221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:18.633906 1221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:42897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:18.634976 1221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:18.636091 1221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:18.645665 1233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:18.645665 1235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:18.645933 1231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:18.646014 1221 server_base.cc:1034] running on GCE node
I20260810 18:53:18.646523 1221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:18.647212 1221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:18.648404 1221 hybrid_clock.cc:648] HybridClock initialized: now 1786387998648370 us; error 44 us; skew 500 ppm
Aug 10 18:53:18 dist-test-slave-68xk krb5kdc[584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:18.651552 1221 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:18.652503 1221 webserver.cc:460] Webserver started at http://127.30.153.131:42031/ using document root <none> and password file <none>
I20260810 18:53:18.653191 1221 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:18.653266 1221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:18.653530 1221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:18.655795 1221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "3ea27f7052bf4f9a92285439c355fe1e"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "9666b33a2c7ef05c48809b55f3a88405"
server_key_iv: "53b3dc140145319ca7f0771644630b5c"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.656457 1221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "3ea27f7052bf4f9a92285439c355fe1e"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "9666b33a2c7ef05c48809b55f3a88405"
server_key_iv: "53b3dc140145319ca7f0771644630b5c"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.660547 1221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:18.663371 1246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:18.664705 1221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:18.664867 1221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "3ea27f7052bf4f9a92285439c355fe1e"
format_stamp: "Formatted at 2026-08-10 18:53:18 on dist-test-slave-68xk"
server_key: "9666b33a2c7ef05c48809b55f3a88405"
server_key_iv: "53b3dc140145319ca7f0771644630b5c"
server_key_version: "encryptionkey@0"
I20260810 18:53:18.664991 1221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:18.683682 1221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:18.686901 1221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:18.687139 1221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:18.688721 1221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:18.688817 1221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:18.688898 1221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:18.688947 1221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:18 dist-test-slave-68xk krb5kdc[584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:18.701529 1221 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:45847
I20260810 18:53:18.701787 1424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:45847 every 8 connection(s)
I20260810 18:53:18.703002 1221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:18.706545 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 1221
I20260810 18:53:18.706691 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:18.706863 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.689907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:53665 (local address 127.30.153.190:42897)
0810 18:53:18.690049 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.690053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.691278 (+ 1225us) server_negotiation.cc:407] Connection header received
0810 18:53:18.692493 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.692498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.692572 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.692653 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:18.694518 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.695578 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.697745 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.698072 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.698162 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.702321 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:18.702326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:18.702329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:18.702364 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:18.704284 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.704980 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.704984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.704986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.705113 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.705511 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.705518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.705521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.705761 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:18.705886 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.706512 (+ 626us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.706670 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:53:18.706995 31334 external_mini_cluster.cc:1428] Setting key bc4c99100654da7662aab17fd982ae2f
I20260810 18:53:18.707727 1260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.690548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42897 (local address 127.30.153.131:53665)
0810 18:53:18.691114 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.691158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.692208 (+ 1050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.693159 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.693168 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.693641 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:18.694314 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.694336 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.695788 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.695794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.697550 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.697566 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.698260 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.698265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.698547 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.699511 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:18.699540 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:18.701908 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:18.704430 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.704438 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.704458 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.704848 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.705222 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.705226 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.705228 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.705389 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.705884 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:18.705892 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:18.706183 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.706401 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.706821 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":165,"threads_started":1}
I20260810 18:53:18.709076 1425 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42897
I20260810 18:53:18.709224 1425 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:18.709852 1425 heartbeater.cc:511] Master 127.30.153.190:42897 requested a full tablet report, sending...
I20260810 18:53:18.711079 660 ts_manager.cc:194] Registered new tserver with Master: 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847)
I20260810 18:53:18.711826 660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:53665
I20260810 18:53:18.719326 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.711654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:56565 (local address 127.30.153.190:42897)
0810 18:53:18.711785 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.711789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.712020 (+ 231us) server_negotiation.cc:407] Connection header received
0810 18:53:18.712171 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.712174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.712219 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.712310 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:18.712775 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.713285 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.714293 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.714626 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.714752 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.715965 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:18.715968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:18.715970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:18.715995 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:18.717463 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.717933 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.717938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.717940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.718029 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.718351 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.718354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.718357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.718623 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:18.718744 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.719032 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.719157 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:53:18.719339 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.711530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42897 (local address 127.30.153.131:56565)
0810 18:53:18.711934 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.711952 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.712051 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.712327 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.712330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.712543 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:18.712659 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.712665 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.713416 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.713419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.714140 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.714151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.714761 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.714764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.714877 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.715434 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:18.715448 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:18.715779 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:18.717592 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.717595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.717598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.717821 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.718139 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:18.718142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:18.718144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:18.718244 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:18.718734 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:18.718737 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:18.718850 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.718971 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.719145 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":147,"threads_started":1}
I20260810 18:53:18.722182 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:53:18 dist-test-slave-68xk krb5kdc[584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:53:18 dist-test-slave-68xk krb5kdc[584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:18.751293 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.742125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.30.153.190:42897)
0810 18:53:18.742262 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.742266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.742279 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:18.742322 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.742324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.742362 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.742500 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:18.743267 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.743801 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.744704 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.744918 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.745014 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.747865 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:18.747868 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:18.747870 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:18.747894 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:18.749283 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.749732 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.749737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.749739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.749841 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:18.750157 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:18.750160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:18.750162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:18.750406 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:18.750627 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.750924 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.751073 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:18.754670 660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:18.757932 660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:18.774246 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.768654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41516 (local address 127.30.153.131:45847)
0810 18:53:18.769278 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.769288 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.769309 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:53:18.769373 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.769377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.769529 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.769710 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:18.769961 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.770945 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.772929 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.773494 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.773684 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.773834 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.773966 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.774074 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":124,"threads_started":1}
I20260810 18:53:18.774240 1460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.767343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:41509 (local address 127.0.0.1:34658)
0810 18:53:18.768057 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.768160 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.768319 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.769255 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.769262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.769286 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:18.769424 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.769432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.771290 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.771296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.772530 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.772540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.773732 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.773737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.773852 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.773885 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.773941 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.774086 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":76,"spinlock_wait_cycles":4480,"thread_start_us":84,"threads_started":1}
I20260810 18:53:18.774585 1458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.767570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38025 (local address 127.0.0.1:35654)
0810 18:53:18.768126 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.768172 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.768318 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.769381 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.769384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.769400 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:18.769491 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.769498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.771180 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.771183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.772716 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.772728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.774120 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.774124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.774221 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.774248 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.774328 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.774398 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":26,"spinlock_wait_cycles":6656,"thread_start_us":108,"threads_started":1}
I20260810 18:53:18.774560 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.767539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.30.153.129:41509)
0810 18:53:18.768701 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.768707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.768731 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:53:18.768799 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.768804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.769077 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.769611 (+ 534us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:18.769773 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.771103 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.772672 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.773550 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.774085 (+ 535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.774212 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.774265 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.774372 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":85,"threads_started":1}
I20260810 18:53:18.774811 1461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.768079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:45847 (local address 127.0.0.1:41516)
0810 18:53:18.768856 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.768871 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.768978 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.769710 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.769713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.769731 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:18.769827 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.769834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.771069 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.771075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.772783 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.772794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.773632 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.773637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.773758 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.773798 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.773929 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.774681 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":106,"thread_start_us":122,"threads_started":1}
I20260810 18:53:18.775471 1462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.768188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.30.153.130:38025)
0810 18:53:18.768651 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.768665 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.768693 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:53:18.768780 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.768786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.769015 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.769210 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:18.769659 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.771037 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.772861 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.773747 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.773887 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.774039 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.774960 (+ 921us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.775061 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":136,"threads_started":1}
I20260810 18:53:18.777992 1321 tablet_service.cc:1467] Processing CreateTablet for tablet 8390547ab9b84b7ead70fe160c84cf93 (DEFAULT_TABLE table=test-table [id=3df710b9b1ac4e07987d85ac46f21f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:18.777946 882 tablet_service.cc:1467] Processing CreateTablet for tablet 8390547ab9b84b7ead70fe160c84cf93 (DEFAULT_TABLE table=test-table [id=3df710b9b1ac4e07987d85ac46f21f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:18.778115 1102 tablet_service.cc:1467] Processing CreateTablet for tablet 8390547ab9b84b7ead70fe160c84cf93 (DEFAULT_TABLE table=test-table [id=3df710b9b1ac4e07987d85ac46f21f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:18.779179 1102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8390547ab9b84b7ead70fe160c84cf93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:18.779199 882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8390547ab9b84b7ead70fe160c84cf93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:18.779685 1321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8390547ab9b84b7ead70fe160c84cf93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:18.786859 1465 tablet_bootstrap.cc:492] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Bootstrap starting.
I20260810 18:53:18.788358 1470 tablet_bootstrap.cc:492] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: Bootstrap starting.
I20260810 18:53:18.788357 1471 tablet_bootstrap.cc:492] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Bootstrap starting.
I20260810 18:53:18.789273 1465 tablet_bootstrap.cc:654] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:18.790304 1465 log.cc:824] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:18.790781 1470 tablet_bootstrap.cc:654] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:18.790779 1471 tablet_bootstrap.cc:654] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:18.791702 1471 log.cc:824] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:18.791702 1470 log.cc:824] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:18.792534 1465 tablet_bootstrap.cc:492] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: No bootstrap required, opened a new log
I20260810 18:53:18.792770 1465 ts_tablet_manager.cc:1397] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:53:18.793829 1471 tablet_bootstrap.cc:492] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: No bootstrap required, opened a new log
I20260810 18:53:18.793829 1470 tablet_bootstrap.cc:492] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: No bootstrap required, opened a new log
I20260810 18:53:18.794034 1470 ts_tablet_manager.cc:1397] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:53:18.794034 1471 ts_tablet_manager.cc:1397] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:53:18.796166 1465 raft_consensus.cc:348] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.796375 1465 raft_consensus.cc:374] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:18.796435 1465 raft_consensus.cc:729] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd573e73fcb74534a223e521a1d4c068, State: Initialized, Role: FOLLOWER
I20260810 18:53:18.796941 1465 consensus_queue.cc:260] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.797715 1206 heartbeater.cc:503] Master 127.30.153.190:42897 was elected leader, sending a full tablet report...
I20260810 18:53:18.798780 1471 raft_consensus.cc:348] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.798995 1465 ts_tablet_manager.cc:1428] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:53:18.799161 1471 raft_consensus.cc:374] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:18.799242 1471 raft_consensus.cc:729] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea27f7052bf4f9a92285439c355fe1e, State: Initialized, Role: FOLLOWER
I20260810 18:53:18.799700 1471 consensus_queue.cc:260] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.799604 1470 raft_consensus.cc:348] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.799857 1470 raft_consensus.cc:374] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:18.799971 1470 raft_consensus.cc:729] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fbadda1fe7e4eb5b877bc3e803f9182, State: Initialized, Role: FOLLOWER
I20260810 18:53:18.800532 1470 consensus_queue.cc:260] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.800840 1471 ts_tablet_manager.cc:1428] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260810 18:53:18.800940 1425 heartbeater.cc:503] Master 127.30.153.190:42897 was elected leader, sending a full tablet report...
I20260810 18:53:18.801218 983 heartbeater.cc:503] Master 127.30.153.190:42897 was elected leader, sending a full tablet report...
I20260810 18:53:18.801525 1470 ts_tablet_manager.cc:1428] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260810 18:53:18.923746 1477 raft_consensus.cc:482] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:18.923954 1477 raft_consensus.cc:504] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.925129 1477 leader_election.cc:288] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fbadda1fe7e4eb5b877bc3e803f9182 (127.30.153.129:41509), 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847)
I20260810 18:53:18.930238 1033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.925483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:41509 (local address 127.30.153.130:49819)
0810 18:53:18.925638 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.925655 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.925764 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.926152 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.926155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.926180 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:18.926340 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.926348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.927508 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.927511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.928918 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.928930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.929837 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.929844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.929974 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.929995 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.930059 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.930120 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:53:18.930357 1459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.925686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:49819 (local address 127.30.153.129:41509)
0810 18:53:18.925839 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.925848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.925886 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:53:18.925970 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.925973 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.926017 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.926134 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:18.926559 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.927364 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.929081 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.929701 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.929811 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.929856 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.930110 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.930213 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:18.931182 915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8390547ab9b84b7ead70fe160c84cf93" candidate_uuid: "dd573e73fcb74534a223e521a1d4c068" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" is_pre_election: true
I20260810 18:53:18.931429 1482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.925805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:45847 (local address 127.30.153.130:32917)
0810 18:53:18.926470 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:18.926486 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:18.926587 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:18.926924 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:18.926927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:18.926947 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:18.927104 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.927110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.927994 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.927998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.929793 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:18.929807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.930968 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.930973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:18.931085 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.931111 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:53:18.931205 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 18:53:18.931280 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":312,"threads_started":1}
I20260810 18:53:18.931489 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.926098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:32917 (local address 127.30.153.131:45847)
0810 18:53:18.926267 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.926272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.926578 (+ 306us) server_negotiation.cc:407] Connection header received
0810 18:53:18.926692 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.926695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.926748 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.926840 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:18.927242 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.927872 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.929945 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.930802 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.930942 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.931004 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.931247 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.931362 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:53:18.931576 915 raft_consensus.cc:2454] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd573e73fcb74534a223e521a1d4c068 in term 0.
I20260810 18:53:18.932204 1021 leader_election.cc:302] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fbadda1fe7e4eb5b877bc3e803f9182, dd573e73fcb74534a223e521a1d4c068; no voters:
I20260810 18:53:18.932214 1351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8390547ab9b84b7ead70fe160c84cf93" candidate_uuid: "dd573e73fcb74534a223e521a1d4c068" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea27f7052bf4f9a92285439c355fe1e" is_pre_election: true
I20260810 18:53:18.932507 1351 raft_consensus.cc:2454] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd573e73fcb74534a223e521a1d4c068 in term 0.
I20260810 18:53:18.932606 1477 raft_consensus.cc:2781] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:18.932708 1477 raft_consensus.cc:482] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:18.932773 1477 raft_consensus.cc:3037] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:18.934010 1477 raft_consensus.cc:504] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.934342 1477 leader_election.cc:288] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [CANDIDATE]: Term 1 election: Requested vote from peers 8fbadda1fe7e4eb5b877bc3e803f9182 (127.30.153.129:41509), 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847)
I20260810 18:53:18.934800 1351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8390547ab9b84b7ead70fe160c84cf93" candidate_uuid: "dd573e73fcb74534a223e521a1d4c068" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea27f7052bf4f9a92285439c355fe1e"
I20260810 18:53:18.934813 915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8390547ab9b84b7ead70fe160c84cf93" candidate_uuid: "dd573e73fcb74534a223e521a1d4c068" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182"
I20260810 18:53:18.934965 1351 raft_consensus.cc:3037] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:18.934973 915 raft_consensus.cc:3037] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:18.936367 915 raft_consensus.cc:2454] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd573e73fcb74534a223e521a1d4c068 in term 1.
I20260810 18:53:18.936487 1351 raft_consensus.cc:2454] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd573e73fcb74534a223e521a1d4c068 in term 1.
I20260810 18:53:18.936755 1021 leader_election.cc:302] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fbadda1fe7e4eb5b877bc3e803f9182, dd573e73fcb74534a223e521a1d4c068; no voters:
I20260810 18:53:18.936966 1477 raft_consensus.cc:2781] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:18.937178 1477 raft_consensus.cc:686] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 1 LEADER]: Becoming Leader. State: Replica: dd573e73fcb74534a223e521a1d4c068, State: Running, Role: LEADER
I20260810 18:53:18.937536 1477 consensus_queue.cc:237] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:18.941082 659 catalog_manager.cc:5495] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd573e73fcb74534a223e521a1d4c068 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } health_report { overall_health: UNKNOWN } } }
W20260810 18:53:18.954073 1426 tablet.cc:2369] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:18.972339 1462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:18.968282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.30.153.130:38025)
0810 18:53:18.968425 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:18.968429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:18.968445 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:18.968722 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:18.968725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:18.968790 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:18.968887 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:18.969176 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.969837 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.970803 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:18.971035 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:18.971127 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:18.971500 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:18.971915 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:18.972008 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:18.972168 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 18:53:18.972228 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 18:53:18.975263 1207 tablet.cc:2369] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:18.979022 1351 raft_consensus.cc:1261] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Refusing update from remote peer dd573e73fcb74534a223e521a1d4c068: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:18.979873 1486 consensus_queue.cc:1035] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:18.980598 915 raft_consensus.cc:1261] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Refusing update from remote peer dd573e73fcb74534a223e521a1d4c068: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:18.981184 1486 consensus_queue.cc:1035] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:18.988583 1490 mvcc.cc:205] Tried to move back new op lower bound from 7317045243808829440 to 7317045243651035136. Current Snapshot: MvccSnapshot[applied={T|T < 7317045243808829440}]
I20260810 18:53:18.990928 1493 mvcc.cc:205] Tried to move back new op lower bound from 7317045243808829440 to 7317045243651035136. Current Snapshot: MvccSnapshot[applied={T|T < 7317045243808829440 or (T in {7317045243808829440})}]
I20260810 18:53:19.001739 660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:19.002038 660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:19.005298 660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158: Sending DeleteTablet for 3 replicas of tablet 8390547ab9b84b7ead70fe160c84cf93
I20260810 18:53:19.006351 882 tablet_service.cc:1514] Processing DeleteTablet for tablet 8390547ab9b84b7ead70fe160c84cf93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:19 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:34658
I20260810 18:53:19.007398 1102 tablet_service.cc:1514] Processing DeleteTablet for tablet 8390547ab9b84b7ead70fe160c84cf93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:19 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:35654
I20260810 18:53:19.008541 1321 tablet_service.cc:1514] Processing DeleteTablet for tablet 8390547ab9b84b7ead70fe160c84cf93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:19 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:41516
I20260810 18:53:19.010072 660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:53:19.010628 1501 tablet_replica.cc:330] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: stopping tablet replica
W20260810 18:53:19.011039 660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:19.011054 1501 raft_consensus.cc:2229] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:19.011551 1501 raft_consensus.cc:2258] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:19.013295 1501 ts_tablet_manager.cc:1905] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:19.014726 1500 tablet_replica.cc:330] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: stopping tablet replica
I20260810 18:53:19.015129 1500 raft_consensus.cc:2229] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:19.015478 1500 raft_consensus.cc:2258] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:19.016606 1102 tablet_service.cc:1467] Processing CreateTablet for tablet 8f28fc6512f14a24b25a3f99323300fd (DEFAULT_TABLE table=test-table [id=24a74d2f24454d07a31a4e400ca654e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:19.016883 1102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f28fc6512f14a24b25a3f99323300fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:19.020083 1500 ts_tablet_manager.cc:1905] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:19.021651 1321 tablet_service.cc:1467] Processing CreateTablet for tablet 8f28fc6512f14a24b25a3f99323300fd (DEFAULT_TABLE table=test-table [id=24a74d2f24454d07a31a4e400ca654e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:19.021812 1501 ts_tablet_manager.cc:1918] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:19.021925 1501 log.cc:1196] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/8390547ab9b84b7ead70fe160c84cf93
I20260810 18:53:19.022079 1321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f28fc6512f14a24b25a3f99323300fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:19.022305 1501 ts_tablet_manager.cc:1939] T 8390547ab9b84b7ead70fe160c84cf93 P dd573e73fcb74534a223e521a1d4c068: Deleting consensus metadata
I20260810 18:53:19.023553 1500 ts_tablet_manager.cc:1918] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:19.023660 1500 log.cc:1196] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/8390547ab9b84b7ead70fe160c84cf93
I20260810 18:53:19.024004 1500 ts_tablet_manager.cc:1939] T 8390547ab9b84b7ead70fe160c84cf93 P 8fbadda1fe7e4eb5b877bc3e803f9182: Deleting consensus metadata
I20260810 18:53:19.025163 1471 tablet_bootstrap.cc:492] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: Bootstrap starting.
I20260810 18:53:19.025496 882 tablet_service.cc:1467] Processing CreateTablet for tablet 8f28fc6512f14a24b25a3f99323300fd (DEFAULT_TABLE table=test-table [id=24a74d2f24454d07a31a4e400ca654e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:19.025811 882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f28fc6512f14a24b25a3f99323300fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:19.026814 1471 tablet_bootstrap.cc:654] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:19.028475 642 catalog_manager.cc:4841] TS dd573e73fcb74534a223e521a1d4c068 (127.30.153.130:38025): tablet 8390547ab9b84b7ead70fe160c84cf93 (table test-table [id=3df710b9b1ac4e07987d85ac46f21f24]) successfully deleted
I20260810 18:53:19.029264 1465 tablet_bootstrap.cc:492] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: Bootstrap starting.
I20260810 18:53:19.029534 1471 tablet_bootstrap.cc:492] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: No bootstrap required, opened a new log
I20260810 18:53:19.029613 1471 ts_tablet_manager.cc:1397] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 18:53:19.030838 1471 raft_consensus.cc:348] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.030959 1471 raft_consensus.cc:374] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:19.030993 1471 raft_consensus.cc:729] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea27f7052bf4f9a92285439c355fe1e, State: Initialized, Role: FOLLOWER
I20260810 18:53:19.031186 1471 consensus_queue.cc:260] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.031297 1465 tablet_bootstrap.cc:654] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:19.032831 1471 ts_tablet_manager.cc:1428] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:19.034849 1465 tablet_bootstrap.cc:492] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: No bootstrap required, opened a new log
I20260810 18:53:19.035385 1465 ts_tablet_manager.cc:1397] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 18:53:19.035645 1470 tablet_bootstrap.cc:492] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: Bootstrap starting.
I20260810 18:53:19.037348 1470 tablet_bootstrap.cc:654] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:19.038386 640 catalog_manager.cc:4841] TS 8fbadda1fe7e4eb5b877bc3e803f9182 (127.30.153.129:41509): tablet 8390547ab9b84b7ead70fe160c84cf93 (table test-table [id=3df710b9b1ac4e07987d85ac46f21f24]) successfully deleted
I20260810 18:53:19.039088 1470 tablet_bootstrap.cc:492] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: No bootstrap required, opened a new log
I20260810 18:53:19.039170 1502 tablet_replica.cc:330] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: stopping tablet replica
I20260810 18:53:19.039196 1470 ts_tablet_manager.cc:1397] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260810 18:53:19.039702 1502 raft_consensus.cc:2229] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:19.040107 1502 raft_consensus.cc:2258] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:19.040086 1470 raft_consensus.cc:348] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.040339 1470 raft_consensus.cc:374] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:19.040386 1470 raft_consensus.cc:729] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fbadda1fe7e4eb5b877bc3e803f9182, State: Initialized, Role: FOLLOWER
I20260810 18:53:19.040623 1470 consensus_queue.cc:260] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.041930 1465 raft_consensus.cc:348] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.042155 1465 raft_consensus.cc:374] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:19.042208 1502 ts_tablet_manager.cc:1905] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:19.042557 1465 raft_consensus.cc:729] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd573e73fcb74534a223e521a1d4c068, State: Initialized, Role: FOLLOWER
I20260810 18:53:19.042867 1470 ts_tablet_manager.cc:1428] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260810 18:53:19.042802 1465 consensus_queue.cc:260] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.043622 1465 ts_tablet_manager.cc:1428] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: Time spent starting tablet: real 0.005s user 0.000s sys 0.001s
I20260810 18:53:19.046633 1502 ts_tablet_manager.cc:1918] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:19.046749 1502 log.cc:1196] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/8390547ab9b84b7ead70fe160c84cf93
I20260810 18:53:19.047118 1502 ts_tablet_manager.cc:1939] T 8390547ab9b84b7ead70fe160c84cf93 P 3ea27f7052bf4f9a92285439c355fe1e: Deleting consensus metadata
I20260810 18:53:19.048805 640 catalog_manager.cc:4841] TS 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847): tablet 8390547ab9b84b7ead70fe160c84cf93 (table test-table [id=3df710b9b1ac4e07987d85ac46f21f24]) successfully deleted
I20260810 18:53:19.140455 1481 raft_consensus.cc:482] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:19.140694 1481 raft_consensus.cc:504] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.142055 1481 leader_election.cc:288] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd573e73fcb74534a223e521a1d4c068 (127.30.153.130:38025), 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847)
I20260810 18:53:19.147571 1463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.142844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:49355 (local address 127.30.153.131:45847)
0810 18:53:19.142982 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.142989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.143298 (+ 309us) server_negotiation.cc:407] Connection header received
0810 18:53:19.143486 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.143488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.143548 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.143616 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:19.143992 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.144495 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.146257 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.146893 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.147042 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.147100 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.147350 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.147438 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:53:19.147688 1509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.142481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:45847 (local address 127.30.153.129:49355)
0810 18:53:19.143215 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:19.143230 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:19.143365 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:19.143725 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:19.143730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:19.143753 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:19.143866 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.143871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.144623 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.144627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.146063 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.146074 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.147088 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.147095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.147201 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.147232 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:53:19.147311 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:53:19.147415 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":184,"threads_started":1}
I20260810 18:53:19.147845 1462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.142835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:55541 (local address 127.30.153.130:38025)
0810 18:53:19.142972 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.142977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.143080 (+ 103us) server_negotiation.cc:407] Connection header received
0810 18:53:19.143322 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.143326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.143388 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.143475 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:19.143832 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.145071 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.146492 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.147004 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.147143 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.147199 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.147591 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.147709 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:19.148356 1512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.142336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38025 (local address 127.30.153.129:55541)
0810 18:53:19.142977 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:19.142993 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:19.143173 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:19.143500 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:19.143503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:19.143517 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:19.143670 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.143679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.145219 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.145222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.146352 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.146360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.147256 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.147262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.147427 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.147449 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:53:19.148078 (+ 629us) client_negotiation.cc:240] Negotiation successful
0810 18:53:19.148163 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":166,"threads_started":1}
I20260810 18:53:19.148401 1351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f28fc6512f14a24b25a3f99323300fd" candidate_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea27f7052bf4f9a92285439c355fe1e" is_pre_election: true
I20260810 18:53:19.148558 1351 raft_consensus.cc:2454] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fbadda1fe7e4eb5b877bc3e803f9182 in term 0.
I20260810 18:53:19.149034 804 leader_election.cc:302] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea27f7052bf4f9a92285439c355fe1e, 8fbadda1fe7e4eb5b877bc3e803f9182; no voters:
I20260810 18:53:19.149318 1481 raft_consensus.cc:2781] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:19.149406 1481 raft_consensus.cc:482] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:19.149456 1481 raft_consensus.cc:3037] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:19.149446 1133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f28fc6512f14a24b25a3f99323300fd" candidate_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd573e73fcb74534a223e521a1d4c068" is_pre_election: true
I20260810 18:53:19.149791 1133 raft_consensus.cc:2454] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fbadda1fe7e4eb5b877bc3e803f9182 in term 0.
I20260810 18:53:19.151131 1481 raft_consensus.cc:504] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.151525 1481 leader_election.cc:288] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [CANDIDATE]: Term 1 election: Requested vote from peers dd573e73fcb74534a223e521a1d4c068 (127.30.153.130:38025), 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847)
I20260810 18:53:19.151988 1133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f28fc6512f14a24b25a3f99323300fd" candidate_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd573e73fcb74534a223e521a1d4c068"
I20260810 18:53:19.152139 1133 raft_consensus.cc:3037] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:19.153640 1133 raft_consensus.cc:2454] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fbadda1fe7e4eb5b877bc3e803f9182 in term 1.
I20260810 18:53:19.153918 1351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f28fc6512f14a24b25a3f99323300fd" candidate_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea27f7052bf4f9a92285439c355fe1e"
I20260810 18:53:19.154053 1351 raft_consensus.cc:3037] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:19.154780 806 leader_election.cc:302] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fbadda1fe7e4eb5b877bc3e803f9182, dd573e73fcb74534a223e521a1d4c068; no voters:
I20260810 18:53:19.155027 1481 raft_consensus.cc:2781] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:19.155282 1351 raft_consensus.cc:2454] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fbadda1fe7e4eb5b877bc3e803f9182 in term 1.
I20260810 18:53:19.155409 1481 raft_consensus.cc:686] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 LEADER]: Becoming Leader. State: Replica: 8fbadda1fe7e4eb5b877bc3e803f9182, State: Running, Role: LEADER
I20260810 18:53:19.155681 1481 consensus_queue.cc:237] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.158649 660 catalog_manager.cc:5495] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fbadda1fe7e4eb5b877bc3e803f9182 (127.30.153.129). New cstate: current_term: 1 leader_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } health_report { overall_health: UNKNOWN } } }
Aug 10 18:53:19 dist-test-slave-68xk krb5kdc[584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387997, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:19.206394 756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.196375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53178 (local address 127.30.153.190:42897)
0810 18:53:19.196523 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.196531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.196882 (+ 351us) server_negotiation.cc:407] Connection header received
0810 18:53:19.197080 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.197083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.197143 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.197231 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:19.198068 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.198723 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.199891 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.200123 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.200215 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.202935 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:19.202938 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:19.202940 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:19.202961 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:19.204544 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:19.205045 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:19.205052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:19.205057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:19.205152 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:19.205446 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:19.205451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:19.205453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:19.205634 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:19.205741 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.206036 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.206237 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":50}
I20260810 18:53:19.206395 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.196361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42897 (local address 127.0.0.1:53178)
0810 18:53:19.196757 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:19.196774 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:19.196901 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:19.197282 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:19.197285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:19.197789 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:19.197907 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.197913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.198835 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.198839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.199743 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.199752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.200297 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.200301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.200400 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.201083 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:19.201098 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:19.202773 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:19.204677 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:19.204689 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:19.204695 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:19.204929 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:19.205255 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:19.205258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:19.205260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:19.205357 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:19.205741 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:19.205749 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:19.205860 (+ 111us) client_negotiation.cc:769] Sending connection context
0810 18:53:19.205978 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:53:19.206177 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":163,"threads_started":1}
I20260810 18:53:19.210888 660 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:53178:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:53:19.211712 660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:19.216913 1102 tablet_service.cc:1467] Processing CreateTablet for tablet da01fd57105240d7a80caf2b87e536a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aea23919fb684289b2c21524738e319e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:19.216913 1321 tablet_service.cc:1467] Processing CreateTablet for tablet da01fd57105240d7a80caf2b87e536a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aea23919fb684289b2c21524738e319e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:19.217350 1102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da01fd57105240d7a80caf2b87e536a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:19.217350 1321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da01fd57105240d7a80caf2b87e536a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:19.220261 1465 tablet_bootstrap.cc:492] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: Bootstrap starting.
I20260810 18:53:19.220607 882 tablet_service.cc:1467] Processing CreateTablet for tablet da01fd57105240d7a80caf2b87e536a4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aea23919fb684289b2c21524738e319e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:19.220920 882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da01fd57105240d7a80caf2b87e536a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:19.221362 1465 tablet_bootstrap.cc:654] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:19.222990 1471 tablet_bootstrap.cc:492] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: Bootstrap starting.
I20260810 18:53:19.223513 1470 tablet_bootstrap.cc:492] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: Bootstrap starting.
I20260810 18:53:19.223568 1465 tablet_bootstrap.cc:492] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: No bootstrap required, opened a new log
I20260810 18:53:19.223659 1465 ts_tablet_manager.cc:1397] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 18:53:19.223912 1471 tablet_bootstrap.cc:654] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:19.224112 1465 raft_consensus.cc:348] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.224241 1465 raft_consensus.cc:374] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:19.224280 1465 raft_consensus.cc:729] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd573e73fcb74534a223e521a1d4c068, State: Initialized, Role: FOLLOWER
I20260810 18:53:19.224416 1465 consensus_queue.cc:260] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.224577 1470 tablet_bootstrap.cc:654] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:19.224722 1477 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.224862 1477 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:19.225070 1465 ts_tablet_manager.cc:1428] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:19.225627 1471 tablet_bootstrap.cc:492] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: No bootstrap required, opened a new log
I20260810 18:53:19.225739 1471 ts_tablet_manager.cc:1397] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:19.225979 1470 tablet_bootstrap.cc:492] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: No bootstrap required, opened a new log
I20260810 18:53:19.226076 1470 ts_tablet_manager.cc:1397] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 18:53:19.226192 1471 raft_consensus.cc:348] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.226326 1471 raft_consensus.cc:374] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:19.226368 1471 raft_consensus.cc:729] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ea27f7052bf4f9a92285439c355fe1e, State: Initialized, Role: FOLLOWER
I20260810 18:53:19.226541 1471 consensus_queue.cc:260] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.226596 1470 raft_consensus.cc:348] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.226725 1470 raft_consensus.cc:374] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:19.226771 1470 raft_consensus.cc:729] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fbadda1fe7e4eb5b877bc3e803f9182, State: Initialized, Role: FOLLOWER
I20260810 18:53:19.226853 1471 ts_tablet_manager.cc:1428] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 18:53:19.226900 1470 consensus_queue.cc:260] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.227171 1470 ts_tablet_manager.cc:1428] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:19.227114 1479 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.227253 1479 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:19.227310 1514 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.227435 1514 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 18:53:19.248078 984 tablet.cc:2369] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:19.550989 1477 raft_consensus.cc:482] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:19.551213 1477 raft_consensus.cc:504] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.551709 1477 leader_election.cc:288] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fbadda1fe7e4eb5b877bc3e803f9182 (127.30.153.129:41509), 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847)
I20260810 18:53:19.552230 915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da01fd57105240d7a80caf2b87e536a4" candidate_uuid: "dd573e73fcb74534a223e521a1d4c068" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" is_pre_election: true
I20260810 18:53:19.552271 1351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da01fd57105240d7a80caf2b87e536a4" candidate_uuid: "dd573e73fcb74534a223e521a1d4c068" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea27f7052bf4f9a92285439c355fe1e" is_pre_election: true
I20260810 18:53:19.552408 915 raft_consensus.cc:2454] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd573e73fcb74534a223e521a1d4c068 in term 0.
I20260810 18:53:19.552425 1351 raft_consensus.cc:2454] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd573e73fcb74534a223e521a1d4c068 in term 0.
I20260810 18:53:19.552773 1021 leader_election.cc:302] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea27f7052bf4f9a92285439c355fe1e, dd573e73fcb74534a223e521a1d4c068; no voters:
I20260810 18:53:19.553014 1477 raft_consensus.cc:2781] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:19.553082 1477 raft_consensus.cc:482] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:19.553110 1477 raft_consensus.cc:3037] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:19.554167 1477 raft_consensus.cc:504] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.554553 1477 leader_election.cc:288] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [CANDIDATE]: Term 1 election: Requested vote from peers 8fbadda1fe7e4eb5b877bc3e803f9182 (127.30.153.129:41509), 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847)
I20260810 18:53:19.554987 915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da01fd57105240d7a80caf2b87e536a4" candidate_uuid: "dd573e73fcb74534a223e521a1d4c068" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182"
I20260810 18:53:19.555053 1351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da01fd57105240d7a80caf2b87e536a4" candidate_uuid: "dd573e73fcb74534a223e521a1d4c068" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ea27f7052bf4f9a92285439c355fe1e"
I20260810 18:53:19.555152 915 raft_consensus.cc:3037] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:19.555169 1351 raft_consensus.cc:3037] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:19.556098 1351 raft_consensus.cc:2454] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd573e73fcb74534a223e521a1d4c068 in term 1.
I20260810 18:53:19.556330 915 raft_consensus.cc:2454] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd573e73fcb74534a223e521a1d4c068 in term 1.
I20260810 18:53:19.556480 1021 leader_election.cc:302] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ea27f7052bf4f9a92285439c355fe1e, dd573e73fcb74534a223e521a1d4c068; no voters:
I20260810 18:53:19.556707 1477 raft_consensus.cc:2781] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:19.556900 1477 raft_consensus.cc:686] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [term 1 LEADER]: Becoming Leader. State: Replica: dd573e73fcb74534a223e521a1d4c068, State: Running, Role: LEADER
I20260810 18:53:19.557121 1477 consensus_queue.cc:237] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } }
I20260810 18:53:19.557500 1514 consensus_queue.cc:1035] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:19.558075 1530 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: TxnStatusTablet state changed. Reason: New leader dd573e73fcb74534a223e521a1d4c068. Latest consensus state: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.558197 1530 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:19.558674 1532 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:53:19.558885 1532 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:53:19.559595 661 catalog_manager.cc:5495] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 reported cstate change: term changed from 0 to 1, leader changed from <none> to dd573e73fcb74534a223e521a1d4c068 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:19.561678 1481 consensus_queue.cc:1035] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:19.593165 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.589705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38025 (local address 127.0.0.1:35672)
0810 18:53:19.589910 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:19.589927 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:19.590047 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:19.590363 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:19.590366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:19.590374 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:19.590513 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.590518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.591255 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.591261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.591885 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.591893 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.592366 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.592370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.592465 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.592512 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:19.592831 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:19.592850 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 18:53:19.592941 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:53:19.593026 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 18:53:19.593165 1462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.589820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35672 (local address 127.30.153.130:38025)
0810 18:53:19.589953 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.589960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.589977 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:53:19.590189 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.590193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.590255 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.590341 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:19.590631 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.591143 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.592021 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.592252 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.592364 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.592668 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:19.592743 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:19.592825 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.592959 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.593017 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:53:19.596539 915 raft_consensus.cc:1261] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Refusing update from remote peer dd573e73fcb74534a223e521a1d4c068: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:19.596549 1351 raft_consensus.cc:1261] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Refusing update from remote peer dd573e73fcb74534a223e521a1d4c068: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:19.597059 1477 consensus_queue.cc:1035] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:19.597261 1530 consensus_queue.cc:1035] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:19.598719 1481 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: TxnStatusTablet state changed. Reason: New leader dd573e73fcb74534a223e521a1d4c068. Latest consensus state: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.598862 1481 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:19.599905 1479 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: TxnStatusTablet state changed. Reason: New leader dd573e73fcb74534a223e521a1d4c068. Latest consensus state: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.600044 1479 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:19.601189 1477 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.601311 1477 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:19.601755 1479 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.601864 1479 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P 3ea27f7052bf4f9a92285439c355fe1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:19.604068 1540 mvcc.cc:205] Tried to move back new op lower bound from 7317045246341906432 to 7317045246187569152. Current Snapshot: MvccSnapshot[applied={T|T < 7317045246341906432 or (T in {7317045246341906432})}]
I20260810 18:53:19.605532 1477 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.605649 1477 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P dd573e73fcb74534a223e521a1d4c068: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:19.606452 1481 tablet_replica.cc:439] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd573e73fcb74534a223e521a1d4c068" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fbadda1fe7e4eb5b877bc3e803f9182" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 41509 } } peers { permanent_uuid: "dd573e73fcb74534a223e521a1d4c068" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38025 } } peers { permanent_uuid: "3ea27f7052bf4f9a92285439c355fe1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 45847 } } }
I20260810 18:53:19.606592 1481 tablet_replica.cc:442] T da01fd57105240d7a80caf2b87e536a4 P 8fbadda1fe7e4eb5b877bc3e803f9182: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:19.622859 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.609122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.30.153.129:41509)
0810 18:53:19.618602 (+ 9480us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.618607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.618620 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:19.618699 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.618701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.618756 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.618884 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:19.619207 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.620149 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.621035 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.621452 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.621575 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.621819 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:19.622456 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:19.622590 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.622627 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.622691 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9404,"thread_start_us":101,"threads_started":1}
I20260810 18:53:19.634366 1462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.631035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:60715 (local address 127.30.153.130:38025)
0810 18:53:19.631263 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.631268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.631383 (+ 115us) server_negotiation.cc:407] Connection header received
0810 18:53:19.631564 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.631568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.631623 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.631704 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:19.631961 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.632447 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.633313 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.633537 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.633646 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.633936 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:19.634010 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:19.634090 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.634181 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.634231 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260810 18:53:19.634379 1562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.630935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38025 (local address 127.30.153.129:60715)
0810 18:53:19.631286 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:19.631302 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:19.631461 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:19.631718 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:19.631720 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:19.631727 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:19.631854 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.631860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.632558 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.632561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.633179 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.633188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.633650 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.633654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.633755 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.633795 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:19.634098 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:19.634107 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:53:19.634170 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 18:53:19.634251 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":117,"threads_started":1}
I20260810 18:53:19.663467 1570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.658803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:41509 (local address 127.30.153.130:53731)
0810 18:53:19.659294 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:19.659306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:19.659437 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:19.659765 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:19.659767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:19.659774 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:19.659870 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.659874 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.661069 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.661076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.661982 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:19.661993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.662779 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.662782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:19.662854 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.662903 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:19.663217 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:19.663221 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:53:19.663268 (+ 47us) client_negotiation.cc:240] Negotiation successful
0810 18:53:19.663335 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":124,"threads_started":1}
I20260810 18:53:19.663478 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.658916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:53731 (local address 127.30.153.129:41509)
0810 18:53:19.659059 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.659064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.659394 (+ 330us) server_negotiation.cc:407] Connection header received
0810 18:53:19.659562 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.659566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.659625 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.659722 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:19.660026 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.660931 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.662224 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.662643 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.662753 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.663041 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:19.663132 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:19.663220 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.663280 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.663339 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:53:19.690487 661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:19.690689 661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:19.693526 661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb9270336f184ffc8b6cbf8e0d818158: Sending DeleteTablet for 3 replicas of tablet 8f28fc6512f14a24b25a3f99323300fd
I20260810 18:53:19.694520 1102 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f28fc6512f14a24b25a3f99323300fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:19 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:35654
I20260810 18:53:19.694609 882 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f28fc6512f14a24b25a3f99323300fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:19 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:34658
I20260810 18:53:19.694944 1321 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f28fc6512f14a24b25a3f99323300fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:19 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:41516
I20260810 18:53:19.695421 1582 tablet_replica.cc:330] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: stopping tablet replica
I20260810 18:53:19.695643 1581 tablet_replica.cc:330] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: stopping tablet replica
I20260810 18:53:19.695834 1582 raft_consensus.cc:2229] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:19.696022 1581 raft_consensus.cc:2229] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:19.696398 1581 raft_consensus.cc:2258] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:19.696595 1582 raft_consensus.cc:2258] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:19.696625 1577 tablet_replica.cc:330] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: stopping tablet replica
I20260810 18:53:19.696780 1577 raft_consensus.cc:2229] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:19.696902 1577 raft_consensus.cc:2258] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:19.697731 1582 ts_tablet_manager.cc:1905] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:19.698024 1577 ts_tablet_manager.cc:1905] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:19.698532 1581 ts_tablet_manager.cc:1905] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:19.701355 1577 ts_tablet_manager.cc:1918] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:19.701445 1577 log.cc:1196] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/8f28fc6512f14a24b25a3f99323300fd
I20260810 18:53:19.701756 1577 ts_tablet_manager.cc:1939] T 8f28fc6512f14a24b25a3f99323300fd P dd573e73fcb74534a223e521a1d4c068: Deleting consensus metadata
I20260810 18:53:19.702590 642 catalog_manager.cc:4841] TS dd573e73fcb74534a223e521a1d4c068 (127.30.153.130:38025): tablet 8f28fc6512f14a24b25a3f99323300fd (table test-table [id=24a74d2f24454d07a31a4e400ca654e3]) successfully deleted
Aug 10 18:53:19 dist-test-slave-68xk krb5kdc[584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786387998, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:19.715740 1581 ts_tablet_manager.cc:1918] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:19.718897 1582 ts_tablet_manager.cc:1918] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:19.719076 1582 log.cc:1196] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/8f28fc6512f14a24b25a3f99323300fd
I20260810 18:53:19.719434 1582 ts_tablet_manager.cc:1939] T 8f28fc6512f14a24b25a3f99323300fd P 3ea27f7052bf4f9a92285439c355fe1e: Deleting consensus metadata
I20260810 18:53:19.718730 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.696327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.30.153.129:41509)
0810 18:53:19.696444 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.696449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.703168 (+ 6719us) server_negotiation.cc:407] Connection header received
0810 18:53:19.703402 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.703406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.703465 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.703526 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:19.704855 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.705725 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.707420 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.707830 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.707922 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.713256 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:19.713271 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:19.713289 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:19.713323 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:19.715895 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:19.716542 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:19.716550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:19.716552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:19.716726 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:19.717149 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:19.717153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:19.717155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:19.717538 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:19.717810 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.718276 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.718513 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:53:19.723428 1581 log.cc:1196] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/8f28fc6512f14a24b25a3f99323300fd
W20260810 18:53:19.723665 977 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34672
I20260810 18:53:19.723814 1581 ts_tablet_manager.cc:1939] T 8f28fc6512f14a24b25a3f99323300fd P 8fbadda1fe7e4eb5b877bc3e803f9182: Deleting consensus metadata
I20260810 18:53:19.724985 640 catalog_manager.cc:4841] TS 3ea27f7052bf4f9a92285439c355fe1e (127.30.153.131:45847): tablet 8f28fc6512f14a24b25a3f99323300fd (table test-table [id=24a74d2f24454d07a31a4e400ca654e3]) successfully deleted
I20260810 18:53:19.726519 640 catalog_manager.cc:4841] TS 8fbadda1fe7e4eb5b877bc3e803f9182 (127.30.153.129:41509): tablet 8f28fc6512f14a24b25a3f99323300fd (table test-table [id=24a74d2f24454d07a31a4e400ca654e3]) successfully deleted
I20260810 18:53:19.741671 1589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:19.727212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.30.153.190:42897)
0810 18:53:19.730702 (+ 3490us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:19.730708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:19.731073 (+ 365us) server_negotiation.cc:407] Connection header received
0810 18:53:19.731316 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:19.731320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:19.731388 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:19.731463 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:19.732703 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.733583 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.734964 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:19.735370 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:19.735451 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:19.737523 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:19.737527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:19.737531 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:19.737571 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:19.739514 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:19.740040 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:19.740044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:19.740046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:19.740172 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:19.740531 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:19.740535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:19.740537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:19.740869 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:19.741024 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:19.741304 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 18:53:19.741456 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3386,"thread_start_us":99,"threads_started":1}
W20260810 18:53:19.742107 661 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53188
I20260810 18:53:19.744082 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 787
I20260810 18:53:19.786701 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 997
I20260810 18:53:19.805167 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 1221
I20260810 18:53:19.827077 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 612
2026-08-10T18:53:19Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3451 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:19 dist-test-slave-68xk krb5kdc[1599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:19 dist-test-slave-68xk krb5kdc[1599](info): set up 2 sockets
Aug 10 18:53:19 dist-test-slave-68xk krb5kdc[1599](info): commencing operation
krb5kdc: starting...
W20260810 18:53:21.161068 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:21 dist-test-slave-68xk krb5kdc[1599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:21.290081 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:40917
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:40917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:21.407224 1619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:21.407516 1619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:21.407557 1619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:21.411826 1619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:21.411898 1619 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:53:21.411916 1619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:21.411934 1619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:21.411950 1619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:21.411993 1619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:21.417507 1619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:40917
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:40917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:21.418735 1619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:21.419817 1619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:21.428776 1628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:21.428820 1632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:21.428753 1626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:21.428896 1619 server_base.cc:1034] running on GCE node
I20260810 18:53:21.429807 1619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:21.430935 1619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:21.432128 1619 hybrid_clock.cc:648] HybridClock initialized: now 1786388001432100 us; error 46 us; skew 500 ppm
Aug 10 18:53:21 dist-test-slave-68xk krb5kdc[1599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:21.435036 1619 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:21.435974 1619 webserver.cc:460] Webserver started at http://127.30.153.190:34667/ using document root <none> and password file <none>
I20260810 18:53:21.436559 1619 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:21.436664 1619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:21.436905 1619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:21.439029 1619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "9acbf5911d834a119807578a5e7dee04"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "61698e080effb88f77a8030b1f34fbfd"
server_key_iv: "a86e0d5081d83c3de3fb49cefd951d2e"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.439617 1619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "9acbf5911d834a119807578a5e7dee04"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "61698e080effb88f77a8030b1f34fbfd"
server_key_iv: "a86e0d5081d83c3de3fb49cefd951d2e"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.443373 1619 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:53:21.445896 1642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:21.447316 1619 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:21.447504 1619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "9acbf5911d834a119807578a5e7dee04"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "61698e080effb88f77a8030b1f34fbfd"
server_key_iv: "a86e0d5081d83c3de3fb49cefd951d2e"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.447649 1619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:21.464812 1619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:21.468387 1619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:21.468627 1619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:21.477252 1619 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:40917
I20260810 18:53:21.477337 1745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:40917 every 8 connection(s)
I20260810 18:53:21.478277 1619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:21.481276 1746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:21.487229 1746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: Bootstrap starting.
I20260810 18:53:21.487437 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 1619
I20260810 18:53:21.487572 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:21.487838 31334 external_mini_cluster.cc:1428] Setting key 4b43a42224d592a55d822921351ed1d7
I20260810 18:53:21.490052 1746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:21.490962 1746 log.cc:824] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:21.493103 1746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: No bootstrap required, opened a new log
Aug 10 18:53:21 dist-test-slave-68xk krb5kdc[1599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:21.497292 1746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9acbf5911d834a119807578a5e7dee04" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40917 } }
I20260810 18:53:21.497622 1746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:21.497706 1746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9acbf5911d834a119807578a5e7dee04, State: Initialized, Role: FOLLOWER
I20260810 18:53:21.498178 1746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9acbf5911d834a119807578a5e7dee04" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40917 } }
I20260810 18:53:21.498404 1746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:21.498522 1746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:21.498656 1746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:21.499817 1746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9acbf5911d834a119807578a5e7dee04" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40917 } }
I20260810 18:53:21.500212 1746 leader_election.cc:302] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9acbf5911d834a119807578a5e7dee04; no voters:
I20260810 18:53:21.500646 1746 leader_election.cc:288] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:21.500823 1753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:21.501102 1753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [term 1 LEADER]: Becoming Leader. State: Replica: 9acbf5911d834a119807578a5e7dee04, State: Running, Role: LEADER
I20260810 18:53:21.501488 1753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9acbf5911d834a119807578a5e7dee04" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40917 } }
I20260810 18:53:21.502154 1746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:21.503770 1759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9acbf5911d834a119807578a5e7dee04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9acbf5911d834a119807578a5e7dee04" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40917 } } }
I20260810 18:53:21.503908 1759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:21.504204 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.489310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49276 (local address 127.30.153.190:40917)
0810 18:53:21.489740 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.489756 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.489791 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:53:21.490545 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.490564 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.490911 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.491295 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:21.492145 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.493119 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.493998 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.494561 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.494742 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.497801 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:21.497808 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:21.497832 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:21.497862 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:21.500166 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.500849 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.500860 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.500867 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.501095 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.501495 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.501501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.501507 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.502111 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:21.502370 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.502790 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.503150 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":147,"threads_started":1}
I20260810 18:53:21.504263 1761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9acbf5911d834a119807578a5e7dee04. Latest consensus state: current_term: 1 leader_uuid: "9acbf5911d834a119807578a5e7dee04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9acbf5911d834a119807578a5e7dee04" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40917 } } }
I20260810 18:53:21.504377 1761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:21.504626 1769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:21.508144 1769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:21.514386 1769 catalog_manager.cc:1347] Generated new cluster ID: 94fadee2af154081b6f47a8c1fc93837
I20260810 18:53:21.514513 1769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:21.526981 1769 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:21.527946 1769 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:21.536804 1769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: Generated new TSK 0
I20260810 18:53:21.537563 1769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:21.573717 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40917
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:21.688259 1787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:21.688575 1787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:21.688660 1787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:21.692720 1787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:21.692828 1787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:21.692909 1787 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:53:21.692955 1787 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:21.693024 1787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:21.698396 1787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:21.699599 1787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:21.700691 1787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:21.711304 1807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:21.711273 1803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:21.711344 1802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:21.711596 1787 server_base.cc:1034] running on GCE node
I20260810 18:53:21.712245 1787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:21.712975 1787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:21.714175 1787 hybrid_clock.cc:648] HybridClock initialized: now 1786388001714160 us; error 32 us; skew 500 ppm
Aug 10 18:53:21 dist-test-slave-68xk krb5kdc[1599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:21.717671 1787 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:21.718803 1787 webserver.cc:460] Webserver started at http://127.30.153.129:44101/ using document root <none> and password file <none>
I20260810 18:53:21.719419 1787 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:21.719520 1787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:21.719767 1787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:21.722100 1787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "dfc6ff59ea6d4676bce6984b87e88a26"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "4beba80be5599ccc78473acfa3937f0d"
server_key_iv: "69707ac623df480548bb4a88c9d93759"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.722739 1787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "dfc6ff59ea6d4676bce6984b87e88a26"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "4beba80be5599ccc78473acfa3937f0d"
server_key_iv: "69707ac623df480548bb4a88c9d93759"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.726853 1787 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:53:21.729424 1818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:21.730863 1787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:21.731024 1787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "dfc6ff59ea6d4676bce6984b87e88a26"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "4beba80be5599ccc78473acfa3937f0d"
server_key_iv: "69707ac623df480548bb4a88c9d93759"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.731197 1787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:21.767310 1787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:21.771175 1787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:21.771416 1787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:21.773020 1787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:21.773072 1787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:21.773150 1787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:21.773181 1787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:21 dist-test-slave-68xk krb5kdc[1599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:21.784996 1787 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:34089
I20260810 18:53:21.785143 2007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:34089 every 8 connection(s)
I20260810 18:53:21.786238 1787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:21.789367 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.774224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:33739 (local address 127.30.153.190:40917)
0810 18:53:21.774365 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.774370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.775511 (+ 1141us) server_negotiation.cc:407] Connection header received
0810 18:53:21.776687 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.776691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.776759 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.776853 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:21.778657 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.779483 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.781460 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.781813 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.781905 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.785337 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:21.785341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:21.785343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:21.785375 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:21.787169 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.787792 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.787795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.787796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.787877 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.788248 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.788250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.788251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.788439 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:21.788518 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.789112 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.789232 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:53:21.790218 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.774768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40917 (local address 127.30.153.129:33739)
0810 18:53:21.775329 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:21.775376 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:21.776399 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:21.777097 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:21.777108 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:21.777664 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:21.778439 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.778466 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.779655 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.779659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.781302 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.781318 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.782000 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.782005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.782245 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.783029 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:21.783055 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:21.785104 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:21.787316 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:21.787321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:21.787353 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:21.787677 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:21.787961 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:21.787963 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:21.787964 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:21.788112 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:21.788567 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:21.788573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:21.788823 (+ 250us) client_negotiation.cc:769] Sending connection context
0810 18:53:21.788984 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 18:53:21.789295 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":142,"threads_started":1}
I20260810 18:53:21.791663 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 1787
I20260810 18:53:21.791791 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:21.791899 2010 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40917
I20260810 18:53:21.792076 2010 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:21.792220 31334 external_mini_cluster.cc:1428] Setting key 61c18221cf73b6e6526d10e589b95527
I20260810 18:53:21.792743 2010 heartbeater.cc:511] Master 127.30.153.190:40917 requested a full tablet report, sending...
I20260810 18:53:21.794394 1666 ts_manager.cc:194] Registered new tserver with Master: dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089)
I20260810 18:53:21.796134 1666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:33739
I20260810 18:53:21.804319 2018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.794280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40917 (local address 127.30.153.129:53173)
0810 18:53:21.794570 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:21.794585 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:21.794665 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:21.797176 (+ 2511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:21.797179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:21.797405 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:21.797520 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.797527 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.798281 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.798285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.798945 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:21.798954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.799648 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.799651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:21.799720 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.800304 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:21.800318 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:21.800663 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:21.802642 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:21.802648 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:21.802650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:21.802877 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:21.803194 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:21.803200 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:21.803204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:21.803294 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:21.803732 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:21.803735 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:21.803837 (+ 102us) client_negotiation.cc:769] Sending connection context
0810 18:53:21.803947 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 18:53:21.804094 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":104,"threads_started":1}
I20260810 18:53:21.804322 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:21.796793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:53173 (local address 127.30.153.190:40917)
0810 18:53:21.796936 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:21.796942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:21.796958 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:21.796998 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:21.797003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:21.797050 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:21.797139 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:21.797668 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.798151 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.799069 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:21.799507 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:21.799633 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:21.800809 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:21.800816 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:21.800822 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:21.800854 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:21.802530 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.803002 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.803006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.803008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.803096 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:21.803390 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:21.803394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:21.803396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:21.803631 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:21.803742 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:21.804002 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 18:53:21.804183 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:21.829700 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40917
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:21.951289 2024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:21.951529 2024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:21.951574 2024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:21.955681 2024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:21.955783 2024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:21.955857 2024 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:53:21.955875 2024 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:21.955914 2024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:21.961247 2024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:21.962292 2024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:21.963508 2024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:21.973862 2034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:21.973862 2033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:21.974237 2024 server_base.cc:1034] running on GCE node
W20260810 18:53:21.974288 2037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:21.974735 2024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:21.975428 2024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:21.976645 2024 hybrid_clock.cc:648] HybridClock initialized: now 1786388001976626 us; error 39 us; skew 500 ppm
Aug 10 18:53:21 dist-test-slave-68xk krb5kdc[1599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:21.979683 2024 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:21.980667 2024 webserver.cc:460] Webserver started at http://127.30.153.130:36089/ using document root <none> and password file <none>
I20260810 18:53:21.981320 2024 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:21.981423 2024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:21.981670 2024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:21.983829 2024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "ee1e515f85bc4029bf78c0ab2aa7833a"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "ffeadf52321d5194da1e0bf61928ed0d"
server_key_iv: "61510647f42acdc5684dd37eb81a132f"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.984411 2024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "ee1e515f85bc4029bf78c0ab2aa7833a"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "ffeadf52321d5194da1e0bf61928ed0d"
server_key_iv: "61510647f42acdc5684dd37eb81a132f"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.988461 2024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:21.991025 2045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:21.992534 2024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:21.992698 2024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "ee1e515f85bc4029bf78c0ab2aa7833a"
format_stamp: "Formatted at 2026-08-10 18:53:21 on dist-test-slave-68xk"
server_key: "ffeadf52321d5194da1e0bf61928ed0d"
server_key_iv: "61510647f42acdc5684dd37eb81a132f"
server_key_version: "encryptionkey@0"
I20260810 18:53:21.992827 2024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:22.012785 2024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:22.016188 2024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:22.016461 2024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:22.018061 2024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:22.018146 2024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:22.018225 2024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:22.018260 2024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:22 dist-test-slave-68xk krb5kdc[1599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:22.029897 2024 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:33045
I20260810 18:53:22.030125 2221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:33045 every 8 connection(s)
I20260810 18:53:22.030970 2024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:22.033404 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.019179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:35605 (local address 127.30.153.190:40917)
0810 18:53:22.019345 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.019351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.020308 (+ 957us) server_negotiation.cc:407] Connection header received
0810 18:53:22.021683 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.021688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.021756 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.021825 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:22.023809 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.024479 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.025845 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.026146 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.026277 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.029159 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:22.029164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:22.029166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:22.029192 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:22.031039 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.031733 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.031739 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.031742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.031870 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.032355 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.032358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.032359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.032543 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:22.032626 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.033156 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.033283 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:53:22.034312 2061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.019582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40917 (local address 127.30.153.130:35605)
0810 18:53:22.020115 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.020175 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.021383 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.022283 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.022292 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.022872 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:22.023649 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.023674 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.024634 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.024638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.025663 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.025674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.026335 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.026340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.026586 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.027187 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:22.027206 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:22.028960 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:22.031182 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.031187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.031201 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.031581 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.031991 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.031995 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.031997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.032246 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.032648 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:22.032654 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:22.032903 (+ 249us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.033072 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.033374 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":123,"threads_started":1}
I20260810 18:53:22.035871 2222 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40917
I20260810 18:53:22.036046 2222 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:22.036674 2222 heartbeater.cc:511] Master 127.30.153.190:40917 requested a full tablet report, sending...
I20260810 18:53:22.037235 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 2024
I20260810 18:53:22.037348 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:22.037678 31334 external_mini_cluster.cc:1428] Setting key d5c0f57818377bbef03421dc3302c727
I20260810 18:53:22.037885 1666 ts_manager.cc:194] Registered new tserver with Master: ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045)
I20260810 18:53:22.038702 1666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:35605
I20260810 18:53:22.047204 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.038767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:46505 (local address 127.30.153.190:40917)
0810 18:53:22.038886 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.038890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.039170 (+ 280us) server_negotiation.cc:407] Connection header received
0810 18:53:22.039289 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.039292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.039341 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.039418 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:22.040016 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.040818 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.042003 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.042247 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.042356 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.043581 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:22.043587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:22.043594 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:22.043626 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:22.045324 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.045872 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.045875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.045876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.045949 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.046282 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.046284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.046285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.046555 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:22.046636 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.046981 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.047078 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 18:53:22.047355 2245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.038730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40917 (local address 127.30.153.130:46505)
0810 18:53:22.039073 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.039088 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.039194 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.039468 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.039471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.039716 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:22.039869 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.039881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.040995 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.041001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.041876 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.041886 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.042372 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.042376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.042518 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.043040 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:22.043051 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:22.043407 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:22.045446 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.045453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.045458 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.045770 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.046056 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.046061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.046065 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.046181 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.046662 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:22.046669 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:22.046793 (+ 124us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.046930 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.047116 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:22.070745 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40917
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:22.195735 2249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:22.196050 2249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:22.196136 2249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:22.200409 2249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:22.200495 2249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:22.200563 2249 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 18:53:22.200605 2249 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:22.200685 2249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:22.206127 2249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:22.207222 2249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:22.208280 2249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:22.218577 2256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:22.218554 2255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:22.218791 2249 server_base.cc:1034] running on GCE node
W20260810 18:53:22.218878 2258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:22.219363 2249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:22.219978 2249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:22.221140 2249 hybrid_clock.cc:648] HybridClock initialized: now 1786388002221101 us; error 49 us; skew 500 ppm
Aug 10 18:53:22 dist-test-slave-68xk krb5kdc[1599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388002, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:22.224130 2249 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:22.225073 2249 webserver.cc:460] Webserver started at http://127.30.153.131:34039/ using document root <none> and password file <none>
I20260810 18:53:22.225720 2249 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:22.225793 2249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:22.226032 2249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:22.228385 2249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "1bac1656e120481aa7e90bbd96d6fd51"
format_stamp: "Formatted at 2026-08-10 18:53:22 on dist-test-slave-68xk"
server_key: "53968faf0eb45cb2a8bcb3e86305a8d4"
server_key_iv: "5884fc5e7a44202c583c00ded6d86d18"
server_key_version: "encryptionkey@0"
I20260810 18:53:22.229084 2249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "1bac1656e120481aa7e90bbd96d6fd51"
format_stamp: "Formatted at 2026-08-10 18:53:22 on dist-test-slave-68xk"
server_key: "53968faf0eb45cb2a8bcb3e86305a8d4"
server_key_iv: "5884fc5e7a44202c583c00ded6d86d18"
server_key_version: "encryptionkey@0"
I20260810 18:53:22.233206 2249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260810 18:53:22.235913 2269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:22.237236 2249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:53:22.237380 2249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "1bac1656e120481aa7e90bbd96d6fd51"
format_stamp: "Formatted at 2026-08-10 18:53:22 on dist-test-slave-68xk"
server_key: "53968faf0eb45cb2a8bcb3e86305a8d4"
server_key_iv: "5884fc5e7a44202c583c00ded6d86d18"
server_key_version: "encryptionkey@0"
I20260810 18:53:22.237495 2249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:22.262689 2249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:22.265755 2249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:22.265918 2249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:22.267373 2249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:22.267470 2249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:22.267550 2249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:22.267596 2249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:22 dist-test-slave-68xk krb5kdc[1599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388002, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:22.279282 2249 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:36085
I20260810 18:53:22.279304 2429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:36085 every 8 connection(s)
I20260810 18:53:22.280251 2249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:22.282893 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.268253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:51245 (local address 127.30.153.190:40917)
0810 18:53:22.268420 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.268426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.269418 (+ 992us) server_negotiation.cc:407] Connection header received
0810 18:53:22.270582 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.270586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.270655 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.270751 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:22.272289 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.273203 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.274939 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.275352 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.275461 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.278762 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:22.278766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:22.278768 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:22.278793 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:22.280483 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.281123 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.281127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.281129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.281229 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.281640 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.281644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.281645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.281935 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:22.282060 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.282605 (+ 545us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.282748 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:53:22.283716 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.268742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40917 (local address 127.30.153.131:51245)
0810 18:53:22.269233 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.269286 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.270293 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.270941 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.270948 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.271402 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:22.272106 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.272125 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.273360 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.273364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.274775 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.274787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.275528 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.275531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.275689 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.276319 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:22.276340 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:22.278535 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:22.280621 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.280627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.280643 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.280999 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.281337 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.281340 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.281342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.281499 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.282058 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:22.282063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:22.282307 (+ 244us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.282540 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.282826 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":156,"threads_started":1}
I20260810 18:53:22.284974 2430 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40917
I20260810 18:53:22.285147 2430 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:22.285733 2430 heartbeater.cc:511] Master 127.30.153.190:40917 requested a full tablet report, sending...
I20260810 18:53:22.287002 1665 ts_manager.cc:194] Registered new tserver with Master: 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085)
I20260810 18:53:22.287853 1665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:51245
I20260810 18:53:22.288738 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 2249
I20260810 18:53:22.288887 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:22.289235 31334 external_mini_cluster.cc:1428] Setting key 79bca585249e7698829699c2492f82fe
I20260810 18:53:22.291584 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:22.298689 2442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.287485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40917 (local address 127.30.153.131:39903)
0810 18:53:22.287786 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.287799 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.287930 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.288268 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.288271 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.288502 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:22.288707 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.288716 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.289805 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.289808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.290653 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.290663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.291174 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.291178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.291291 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.294280 (+ 2989us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:22.294293 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:22.294748 (+ 455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:22.296773 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.296779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.296782 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.297105 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.297431 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.297436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.297438 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.297536 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.298099 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:22.298102 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:22.298203 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.298318 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.298534 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":102,"threads_started":1}
I20260810 18:53:22.298893 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.287586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:39903 (local address 127.30.153.190:40917)
0810 18:53:22.287729 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.287733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.287871 (+ 138us) server_negotiation.cc:407] Connection header received
0810 18:53:22.288079 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.288083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.288137 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.288242 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:22.288853 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.289655 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.290785 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.291037 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.291168 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.294906 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:22.294910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:22.294913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:22.294943 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:22.296587 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.297227 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.297231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.297233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.297321 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.297640 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.297645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.297647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.297937 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:22.298059 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.298461 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.298636 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 18:53:22 dist-test-slave-68xk krb5kdc[1599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:53:22 dist-test-slave-68xk krb5kdc[1599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:22.322640 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.313122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.30.153.190:40917)
0810 18:53:22.313261 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.313266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.313279 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:22.313321 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.313324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.313363 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.313441 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:22.314119 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.314795 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.315793 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.316047 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.316140 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.319040 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:22.319043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:22.319045 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:22.319071 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:22.320712 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.321196 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.321236 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.321238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.321324 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.321644 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.321648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.321649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.321859 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:22.322018 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.322283 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.322441 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:53:22.326016 1665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:22.329181 1665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:22.343973 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.338498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.30.153.129:34089)
0810 18:53:22.339167 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.339176 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.339206 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:53:22.339301 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.339317 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.339483 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.339659 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:22.339987 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.340764 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.342478 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.343305 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.343438 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.343582 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.343747 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.343839 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":91,"threads_started":1}
I20260810 18:53:22.343973 2456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.338204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:34089 (local address 127.0.0.1:40924)
0810 18:53:22.338794 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.338847 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.339009 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.339659 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.339665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.339688 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:22.339816 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.339831 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.340895 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.340898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.342298 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.342309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.343436 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.343440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.343558 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.343599 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.343703 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.343825 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":95,"threads_started":1}
I20260810 18:53:22.344727 2459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.338204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:36085 (local address 127.0.0.1:33156)
0810 18:53:22.339020 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.339033 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.339112 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.340333 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.340336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.340353 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:22.340472 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.340481 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.341880 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.341883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.343028 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.343039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.344306 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.344312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.344416 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.344439 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.344505 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.344569 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":540,"thread_start_us":110,"threads_started":1}
I20260810 18:53:22.345240 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.338479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.30.153.131:36085)
0810 18:53:22.339816 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.339828 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.339853 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:53:22.339930 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.339935 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.340196 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.340355 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:22.340598 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.341744 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.343162 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.343960 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.344101 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.344242 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.344977 (+ 735us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.345098 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1236,"thread_start_us":187,"threads_started":1}
I20260810 18:53:22.346047 2460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.339009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:33045 (local address 127.0.0.1:35604)
0810 18:53:22.339452 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.339467 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.339563 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.341472 (+ 1909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.341475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.341493 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:22.341615 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.341623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.342947 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.342952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.344251 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.344265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.345673 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.345677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.345774 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.345798 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.345848 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.345917 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":89,"threads_started":1}
I20260810 18:53:22.346647 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.339388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35604 (local address 127.30.153.130:33045)
0810 18:53:22.341089 (+ 1701us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.341105 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.341135 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:53:22.341207 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.341210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.341339 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.341493 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:22.341744 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.342837 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.344391 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.345274 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.346252 (+ 978us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.346342 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.346381 (+ 39us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.346535 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1567,"thread_start_us":140,"threads_started":1}
I20260810 18:53:22.347255 1896 tablet_service.cc:1467] Processing CreateTablet for tablet d5f27006d0004d2a957b15be22e9d7a7 (DEFAULT_TABLE table=test-table [id=be6f3644df2342c2b3fb65d5f7e1db22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:22.348016 2337 tablet_service.cc:1467] Processing CreateTablet for tablet d5f27006d0004d2a957b15be22e9d7a7 (DEFAULT_TABLE table=test-table [id=be6f3644df2342c2b3fb65d5f7e1db22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:22.348451 1896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f27006d0004d2a957b15be22e9d7a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.349061 2337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f27006d0004d2a957b15be22e9d7a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.353354 2120 tablet_service.cc:1467] Processing CreateTablet for tablet d5f27006d0004d2a957b15be22e9d7a7 (DEFAULT_TABLE table=test-table [id=be6f3644df2342c2b3fb65d5f7e1db22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:22.354591 2120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f27006d0004d2a957b15be22e9d7a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.356720 2466 tablet_bootstrap.cc:492] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Bootstrap starting.
I20260810 18:53:22.358526 2467 tablet_bootstrap.cc:492] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Bootstrap starting.
I20260810 18:53:22.359819 2466 tablet_bootstrap.cc:654] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.360818 2467 tablet_bootstrap.cc:654] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.360823 2466 log.cc:824] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:22.361630 2467 log.cc:824] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:22.362733 2471 tablet_bootstrap.cc:492] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Bootstrap starting.
I20260810 18:53:22.363408 2466 tablet_bootstrap.cc:492] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: No bootstrap required, opened a new log
I20260810 18:53:22.363632 2467 tablet_bootstrap.cc:492] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: No bootstrap required, opened a new log
I20260810 18:53:22.363657 2466 ts_tablet_manager.cc:1397] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:53:22.363849 2467 ts_tablet_manager.cc:1397] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260810 18:53:22.364995 2471 tablet_bootstrap.cc:654] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.365847 2471 log.cc:824] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:22.368062 2471 tablet_bootstrap.cc:492] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: No bootstrap required, opened a new log
I20260810 18:53:22.367913 2466 raft_consensus.cc:348] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.368175 2466 raft_consensus.cc:374] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.368248 2466 raft_consensus.cc:729] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc6ff59ea6d4676bce6984b87e88a26, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.368269 2471 ts_tablet_manager.cc:1397] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 18:53:22.368211 2467 raft_consensus.cc:348] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.368469 2467 raft_consensus.cc:374] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.368528 2467 raft_consensus.cc:729] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bac1656e120481aa7e90bbd96d6fd51, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.368824 2466 consensus_queue.cc:260] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.368945 2467 consensus_queue.cc:260] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.369664 2430 heartbeater.cc:503] Master 127.30.153.190:40917 was elected leader, sending a full tablet report...
I20260810 18:53:22.369910 2467 ts_tablet_manager.cc:1428] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260810 18:53:22.369920 2466 ts_tablet_manager.cc:1428] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:53:22.370285 2010 heartbeater.cc:503] Master 127.30.153.190:40917 was elected leader, sending a full tablet report...
I20260810 18:53:22.372098 2471 raft_consensus.cc:348] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.372372 2471 raft_consensus.cc:374] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.372444 2471 raft_consensus.cc:729] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1e515f85bc4029bf78c0ab2aa7833a, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.372951 2471 consensus_queue.cc:260] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.373953 2471 ts_tablet_manager.cc:1428] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260810 18:53:22.374307 2222 heartbeater.cc:503] Master 127.30.153.190:40917 was elected leader, sending a full tablet report...
W20260810 18:53:22.531431 2433 tablet.cc:2369] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:22.532289 2223 tablet.cc:2369] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:22.537798 2011 tablet.cc:2369] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:22.612908 2477 raft_consensus.cc:482] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:22.613133 2477 raft_consensus.cc:504] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.614344 2477 leader_election.cc:288] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089), ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045)
I20260810 18:53:22.618633 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.614674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:34089 (local address 127.30.153.131:42441)
0810 18:53:22.614950 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.614969 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.615076 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.615404 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.615408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.615432 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:22.615576 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.615585 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.616411 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.616414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.617367 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.617378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.618137 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.618139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.618292 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.618324 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.618403 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.618525 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1}
I20260810 18:53:22.618693 2457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.614799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:42441 (local address 127.30.153.129:34089)
0810 18:53:22.614947 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.614951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.615022 (+ 71us) server_negotiation.cc:407] Connection header received
0810 18:53:22.615211 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.615214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.615278 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.615382 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:22.615704 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.616270 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.617524 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.618008 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.618136 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.618192 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.618448 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.618534 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:22.619354 1931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f27006d0004d2a957b15be22e9d7a7" candidate_uuid: "1bac1656e120481aa7e90bbd96d6fd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" is_pre_election: true
I20260810 18:53:22.619450 2464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.615198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:49655 (local address 127.30.153.130:33045)
0810 18:53:22.615349 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.615355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.615512 (+ 157us) server_negotiation.cc:407] Connection header received
0810 18:53:22.615599 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.615601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.615665 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.615751 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:22.616408 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.616903 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.618049 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.618669 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.618801 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.618855 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.619271 (+ 416us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.619346 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:53:22.619470 2479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.615027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:33045 (local address 127.30.153.131:49655)
0810 18:53:22.615403 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.615416 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.615511 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.615769 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.615771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.615788 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:22.616254 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.616260 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.617017 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.617021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.617908 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.617915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.618959 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.618962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.619062 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.619095 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.619250 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.619321 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":1,"thread_start_us":165,"threads_started":1}
I20260810 18:53:22.619722 1931 raft_consensus.cc:2454] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bac1656e120481aa7e90bbd96d6fd51 in term 0.
I20260810 18:53:22.620327 2273 leader_election.cc:302] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bac1656e120481aa7e90bbd96d6fd51, dfc6ff59ea6d4676bce6984b87e88a26; no voters:
I20260810 18:53:22.620353 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f27006d0004d2a957b15be22e9d7a7" candidate_uuid: "1bac1656e120481aa7e90bbd96d6fd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" is_pre_election: true
I20260810 18:53:22.620648 2150 raft_consensus.cc:2454] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bac1656e120481aa7e90bbd96d6fd51 in term 0.
I20260810 18:53:22.620635 2477 raft_consensus.cc:2781] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:22.620746 2477 raft_consensus.cc:482] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:22.620811 2477 raft_consensus.cc:3037] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:22.621984 2477 raft_consensus.cc:504] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.622347 2477 leader_election.cc:288] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [CANDIDATE]: Term 1 election: Requested vote from peers dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089), ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045)
I20260810 18:53:22.622918 1931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f27006d0004d2a957b15be22e9d7a7" candidate_uuid: "1bac1656e120481aa7e90bbd96d6fd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc6ff59ea6d4676bce6984b87e88a26"
I20260810 18:53:22.623005 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f27006d0004d2a957b15be22e9d7a7" candidate_uuid: "1bac1656e120481aa7e90bbd96d6fd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a"
I20260810 18:53:22.623057 1931 raft_consensus.cc:3037] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:22.623207 2150 raft_consensus.cc:3037] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:22.624300 1931 raft_consensus.cc:2454] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bac1656e120481aa7e90bbd96d6fd51 in term 1.
I20260810 18:53:22.624320 2150 raft_consensus.cc:2454] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bac1656e120481aa7e90bbd96d6fd51 in term 1.
I20260810 18:53:22.624648 2273 leader_election.cc:302] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bac1656e120481aa7e90bbd96d6fd51, ee1e515f85bc4029bf78c0ab2aa7833a; no voters:
I20260810 18:53:22.624930 2477 raft_consensus.cc:2781] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:22.625264 2477 raft_consensus.cc:686] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 LEADER]: Becoming Leader. State: Replica: 1bac1656e120481aa7e90bbd96d6fd51, State: Running, Role: LEADER
I20260810 18:53:22.625651 2477 consensus_queue.cc:237] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } }
I20260810 18:53:22.628988 1666 catalog_manager.cc:5495] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131). New cstate: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } health_report { overall_health: UNKNOWN } } }
Aug 10 18:53:22 dist-test-slave-68xk krb5kdc[1599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388001, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:22.665160 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.656257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40917 (local address 127.0.0.1:49282)
0810 18:53:22.656635 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.656654 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.656814 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.657077 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.657080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.657550 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:22.657645 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.657651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.658435 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.658440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.658994 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.659002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.659500 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.659503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.659585 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.660149 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:22.660161 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:22.661751 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:22.663497 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.663507 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.663513 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.663767 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.664109 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:22.664112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:22.664114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:22.664206 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:22.664605 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:22.664610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:22.664724 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.664823 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.664947 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":136,"threads_started":1}
I20260810 18:53:22.665195 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.656396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49282 (local address 127.30.153.190:40917)
0810 18:53:22.656581 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.656586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.656750 (+ 164us) server_negotiation.cc:407] Connection header received
0810 18:53:22.656916 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.656920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.656965 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.657056 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:22.657782 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.658302 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.659117 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.659330 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.659492 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.661891 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:22.661894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:22.661896 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:22.661918 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:22.663385 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.663895 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.663898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.663899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.663984 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:22.664304 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:22.664306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:22.664308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:22.664494 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:22.664578 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.664897 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.665003 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":82}
I20260810 18:53:22.669793 1666 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:49282:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:53:22.670662 1666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:22.675834 2120 tablet_service.cc:1467] Processing CreateTablet for tablet 0547cc097e214f80a824cf62279fa208 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9aa5f1ed9d2d4ad68ff1e2a9fabe8f05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:22.675835 2337 tablet_service.cc:1467] Processing CreateTablet for tablet 0547cc097e214f80a824cf62279fa208 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9aa5f1ed9d2d4ad68ff1e2a9fabe8f05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:22.675873 1896 tablet_service.cc:1467] Processing CreateTablet for tablet 0547cc097e214f80a824cf62279fa208 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9aa5f1ed9d2d4ad68ff1e2a9fabe8f05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:22.676626 2337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0547cc097e214f80a824cf62279fa208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.676636 1896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0547cc097e214f80a824cf62279fa208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.676635 2120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0547cc097e214f80a824cf62279fa208. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.679745 2471 tablet_bootstrap.cc:492] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: Bootstrap starting.
I20260810 18:53:22.679795 2467 tablet_bootstrap.cc:492] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: Bootstrap starting.
I20260810 18:53:22.680625 2471 tablet_bootstrap.cc:654] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.680678 2467 tablet_bootstrap.cc:654] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.681264 2466 tablet_bootstrap.cc:492] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: Bootstrap starting.
I20260810 18:53:22.682011 2471 tablet_bootstrap.cc:492] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: No bootstrap required, opened a new log
I20260810 18:53:22.682125 2471 ts_tablet_manager.cc:1397] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260810 18:53:22.682356 2466 tablet_bootstrap.cc:654] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.682744 2467 tablet_bootstrap.cc:492] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: No bootstrap required, opened a new log
I20260810 18:53:22.682852 2467 ts_tablet_manager.cc:1397] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260810 18:53:22.682775 2471 raft_consensus.cc:348] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.682901 2471 raft_consensus.cc:374] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.682950 2471 raft_consensus.cc:729] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1e515f85bc4029bf78c0ab2aa7833a, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.683106 2471 consensus_queue.cc:260] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.683421 2471 ts_tablet_manager.cc:1428] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 18:53:22.683364 2467 raft_consensus.cc:348] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.683482 2467 raft_consensus.cc:374] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.683530 2467 raft_consensus.cc:729] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bac1656e120481aa7e90bbd96d6fd51, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.683682 2467 consensus_queue.cc:260] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.683693 2478 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.683833 2478 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:22.683997 2467 ts_tablet_manager.cc:1428] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 18:53:22.684026 2466 tablet_bootstrap.cc:492] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: No bootstrap required, opened a new log
I20260810 18:53:22.684118 2466 ts_tablet_manager.cc:1397] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:22.684625 2477 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.684641 2466 raft_consensus.cc:348] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.684788 2477 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:22.684763 2466 raft_consensus.cc:374] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.684849 2466 raft_consensus.cc:729] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc6ff59ea6d4676bce6984b87e88a26, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.684999 2466 consensus_queue.cc:260] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.685320 2466 ts_tablet_manager.cc:1428] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:22.685590 2476 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.685748 2476 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:22.797034 2477 raft_consensus.cc:482] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:22.797178 2477 raft_consensus.cc:504] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.797649 2477 leader_election.cc:288] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089), ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045)
I20260810 18:53:22.798282 1931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0547cc097e214f80a824cf62279fa208" candidate_uuid: "1bac1656e120481aa7e90bbd96d6fd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" is_pre_election: true
I20260810 18:53:22.798311 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0547cc097e214f80a824cf62279fa208" candidate_uuid: "1bac1656e120481aa7e90bbd96d6fd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" is_pre_election: true
I20260810 18:53:22.798467 1931 raft_consensus.cc:2454] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bac1656e120481aa7e90bbd96d6fd51 in term 0.
I20260810 18:53:22.798520 2150 raft_consensus.cc:2454] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bac1656e120481aa7e90bbd96d6fd51 in term 0.
I20260810 18:53:22.798772 2273 leader_election.cc:302] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bac1656e120481aa7e90bbd96d6fd51, dfc6ff59ea6d4676bce6984b87e88a26; no voters:
I20260810 18:53:22.799000 2477 raft_consensus.cc:2781] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:22.799080 2477 raft_consensus.cc:482] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:22.799135 2477 raft_consensus.cc:3037] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:22.800136 2477 raft_consensus.cc:504] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.800470 2477 leader_election.cc:288] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [CANDIDATE]: Term 1 election: Requested vote from peers dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089), ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045)
I20260810 18:53:22.800998 1931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0547cc097e214f80a824cf62279fa208" candidate_uuid: "1bac1656e120481aa7e90bbd96d6fd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc6ff59ea6d4676bce6984b87e88a26"
I20260810 18:53:22.801031 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0547cc097e214f80a824cf62279fa208" candidate_uuid: "1bac1656e120481aa7e90bbd96d6fd51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a"
I20260810 18:53:22.801122 2150 raft_consensus.cc:3037] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:22.801136 1931 raft_consensus.cc:3037] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:22.802158 2150 raft_consensus.cc:2454] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bac1656e120481aa7e90bbd96d6fd51 in term 1.
I20260810 18:53:22.802201 1931 raft_consensus.cc:2454] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bac1656e120481aa7e90bbd96d6fd51 in term 1.
I20260810 18:53:22.802515 2273 leader_election.cc:302] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bac1656e120481aa7e90bbd96d6fd51, ee1e515f85bc4029bf78c0ab2aa7833a; no voters:
I20260810 18:53:22.802699 2477 raft_consensus.cc:2781] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:22.802794 2477 raft_consensus.cc:686] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 LEADER]: Becoming Leader. State: Replica: 1bac1656e120481aa7e90bbd96d6fd51, State: Running, Role: LEADER
I20260810 18:53:22.802956 2477 consensus_queue.cc:237] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.803684 2482 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: TxnStatusTablet state changed. Reason: New leader 1bac1656e120481aa7e90bbd96d6fd51. Latest consensus state: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.803797 2482 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:22.804071 2497 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:53:22.804309 2497 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:53:22.804859 1666 catalog_manager.cc:5495] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131). New cstate: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } health_report { overall_health: HEALTHY } } }
I20260810 18:53:22.838770 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.834550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.30.153.131:36085)
0810 18:53:22.834685 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.834689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.834704 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:53:22.834853 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.834856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.834908 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.834990 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:22.835316 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.836163 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.837176 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.837463 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.837573 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.837891 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:22.838318 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:22.838470 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.838559 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.838626 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:53:22.838770 2492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.834404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:36085 (local address 127.0.0.1:33160)
0810 18:53:22.834577 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.834594 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.834702 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.835023 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.835026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.835033 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:22.835177 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.835184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.836297 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.836303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.837028 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.837038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.837573 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.837576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.837670 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.837713 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:22.838451 (+ 738us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:22.838470 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.838538 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.838627 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 18:53:22.845012 1931 raft_consensus.cc:1261] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Refusing update from remote peer 1bac1656e120481aa7e90bbd96d6fd51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:22.845021 2150 raft_consensus.cc:1261] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Refusing update from remote peer 1bac1656e120481aa7e90bbd96d6fd51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:22.845772 2482 consensus_queue.cc:1035] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:22.845947 2477 consensus_queue.cc:1035] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:22.849880 2476 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: TxnStatusTablet state changed. Reason: New leader 1bac1656e120481aa7e90bbd96d6fd51. Latest consensus state: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.850026 2476 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:22.850641 2478 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: TxnStatusTablet state changed. Reason: New leader 1bac1656e120481aa7e90bbd96d6fd51. Latest consensus state: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.850775 2478 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:22.853782 2482 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.853809 2478 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.853910 2482 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:22.853924 2478 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P ee1e515f85bc4029bf78c0ab2aa7833a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:22.854082 2505 mvcc.cc:205] Tried to move back new op lower bound from 7317045259645775872 to 7317045259482460160. Current Snapshot: MvccSnapshot[applied={T|T < 7317045259645775872}]
I20260810 18:53:22.854739 2477 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.854846 2477 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P 1bac1656e120481aa7e90bbd96d6fd51: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:22.855854 2476 tablet_replica.cc:439] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1bac1656e120481aa7e90bbd96d6fd51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } } }
I20260810 18:53:22.855969 2476 tablet_replica.cc:442] T 0547cc097e214f80a824cf62279fa208 P dfc6ff59ea6d4676bce6984b87e88a26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:22.857215 2508 mvcc.cc:205] Tried to move back new op lower bound from 7317045259645775872 to 7317045259482460160. Current Snapshot: MvccSnapshot[applied={T|T < 7317045259645775872}]
I20260810 18:53:22.867749 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.863987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.30.153.131:36085)
0810 18:53:22.864118 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.864123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.864348 (+ 225us) server_negotiation.cc:407] Connection header received
0810 18:53:22.864438 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.864441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.864492 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.864582 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:22.864828 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.865620 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.866536 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.866877 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.866983 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.867303 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:22.867381 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:22.867452 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.867568 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.867619 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:53:22.878787 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.875198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:36227 (local address 127.30.153.131:36085)
0810 18:53:22.875329 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:22.875332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:22.875497 (+ 165us) server_negotiation.cc:407] Connection header received
0810 18:53:22.875621 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:22.875624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:22.875662 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:22.875729 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:22.875999 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.876488 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.877692 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.877894 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.877982 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.878334 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:22.878395 (+ 61us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:22.878504 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:22.878629 (+ 125us) server_negotiation.cc:299] Negotiation successful
0810 18:53:22.878676 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":47}
I20260810 18:53:22.878854 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:22.875068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:36085 (local address 127.30.153.131:36227)
0810 18:53:22.875407 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:22.875422 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:22.875523 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:22.875762 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:22.875765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:22.875773 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:22.875899 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.875905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.876599 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.876602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.877553 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:22.877574 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:22.878007 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:22.878011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:22.878106 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:22.878158 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:22.878522 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:22.878529 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:53:22.878607 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 18:53:22.878687 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":79,"threads_started":1}
I20260810 18:53:22.885917 2150 raft_consensus.cc:1261] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Refusing update from remote peer 1bac1656e120481aa7e90bbd96d6fd51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:22.885918 1931 raft_consensus.cc:1261] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Refusing update from remote peer 1bac1656e120481aa7e90bbd96d6fd51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:22.886381 2482 consensus_queue.cc:1035] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:22.886571 2514 consensus_queue.cc:1035] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:22.893266 2506 mvcc.cc:205] Tried to move back new op lower bound from 7317045259817213952 to 7317045258757488640. Current Snapshot: MvccSnapshot[applied={T|T < 7317045259817213952}]
I20260810 18:53:22.928728 1666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:22.928893 1666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:22.931821 1666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: Sending DeleteTablet for 3 replicas of tablet d5f27006d0004d2a957b15be22e9d7a7
I20260810 18:53:22.932489 2336 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f27006d0004d2a957b15be22e9d7a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:22 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33156
I20260810 18:53:22.932636 2120 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f27006d0004d2a957b15be22e9d7a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:22 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:35604
I20260810 18:53:22.933081 2527 tablet_replica.cc:330] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: stopping tablet replica
I20260810 18:53:22.933070 1896 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f27006d0004d2a957b15be22e9d7a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:22 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:40924
I20260810 18:53:22.933416 2527 raft_consensus.cc:2229] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:22.933646 2531 tablet_replica.cc:330] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: stopping tablet replica
I20260810 18:53:22.933827 2527 raft_consensus.cc:2258] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:22.934100 2531 raft_consensus.cc:2229] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:22.934319 1666 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:53:22.934460 2531 raft_consensus.cc:2258] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:22.934553 2528 tablet_replica.cc:330] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: stopping tablet replica
W20260810 18:53:22.934808 1666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:22.934832 2528 raft_consensus.cc:2229] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:22.935043 2528 raft_consensus.cc:2258] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:22.936404 2531 ts_tablet_manager.cc:1905] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:22.936406 2528 ts_tablet_manager.cc:1905] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:22.939853 2531 ts_tablet_manager.cc:1918] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:22.940035 2531 log.cc:1196] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/d5f27006d0004d2a957b15be22e9d7a7
I20260810 18:53:22.940469 2531 ts_tablet_manager.cc:1939] T d5f27006d0004d2a957b15be22e9d7a7 P dfc6ff59ea6d4676bce6984b87e88a26: Deleting consensus metadata
I20260810 18:53:22.940550 2120 tablet_service.cc:1467] Processing CreateTablet for tablet 5b1fa843ad1e49f493b50f48d3fe0eab (DEFAULT_TABLE table=test-table [id=6a1349adc0db49d9a6e21f8166734b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:22.940716 2336 tablet_service.cc:1467] Processing CreateTablet for tablet 5b1fa843ad1e49f493b50f48d3fe0eab (DEFAULT_TABLE table=test-table [id=6a1349adc0db49d9a6e21f8166734b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:22.940831 2120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b1fa843ad1e49f493b50f48d3fe0eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.941030 2336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b1fa843ad1e49f493b50f48d3fe0eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.943454 2527 ts_tablet_manager.cc:1905] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:22.943512 2471 tablet_bootstrap.cc:492] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: Bootstrap starting.
I20260810 18:53:22.943493 2467 tablet_bootstrap.cc:492] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: Bootstrap starting.
I20260810 18:53:22.944339 2467 tablet_bootstrap.cc:654] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.944505 2471 tablet_bootstrap.cc:654] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.945428 2467 tablet_bootstrap.cc:492] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: No bootstrap required, opened a new log
I20260810 18:53:22.945523 2467 ts_tablet_manager.cc:1397] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260810 18:53:22.945636 1896 tablet_service.cc:1467] Processing CreateTablet for tablet 5b1fa843ad1e49f493b50f48d3fe0eab (DEFAULT_TABLE table=test-table [id=6a1349adc0db49d9a6e21f8166734b91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:22.945914 1896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b1fa843ad1e49f493b50f48d3fe0eab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:22.945984 2467 raft_consensus.cc:348] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.946107 2467 raft_consensus.cc:374] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.946165 2467 raft_consensus.cc:729] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bac1656e120481aa7e90bbd96d6fd51, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.946305 2467 consensus_queue.cc:260] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.946619 2467 ts_tablet_manager.cc:1428] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 18:53:22.948240 2527 ts_tablet_manager.cc:1918] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:22.948357 2466 tablet_bootstrap.cc:492] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: Bootstrap starting.
I20260810 18:53:22.948391 2527 log.cc:1196] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/d5f27006d0004d2a957b15be22e9d7a7
I20260810 18:53:22.948822 2527 ts_tablet_manager.cc:1939] T d5f27006d0004d2a957b15be22e9d7a7 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting consensus metadata
I20260810 18:53:22.948818 1645 catalog_manager.cc:4841] TS dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089): tablet d5f27006d0004d2a957b15be22e9d7a7 (table test-table [id=be6f3644df2342c2b3fb65d5f7e1db22]) successfully deleted
I20260810 18:53:22.949419 2466 tablet_bootstrap.cc:654] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:22.950235 1644 catalog_manager.cc:4841] TS 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085): tablet d5f27006d0004d2a957b15be22e9d7a7 (table test-table [id=be6f3644df2342c2b3fb65d5f7e1db22]) successfully deleted
I20260810 18:53:22.951184 2471 tablet_bootstrap.cc:492] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: No bootstrap required, opened a new log
I20260810 18:53:22.951293 2471 ts_tablet_manager.cc:1397] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260810 18:53:22.951594 2528 ts_tablet_manager.cc:1918] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:22.951829 2471 raft_consensus.cc:348] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.951964 2528 log.cc:1196] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/d5f27006d0004d2a957b15be22e9d7a7
I20260810 18:53:22.952450 2528 ts_tablet_manager.cc:1939] T d5f27006d0004d2a957b15be22e9d7a7 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting consensus metadata
I20260810 18:53:22.951982 2471 raft_consensus.cc:374] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.952905 2471 raft_consensus.cc:729] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1e515f85bc4029bf78c0ab2aa7833a, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.953075 2471 consensus_queue.cc:260] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.953433 2471 ts_tablet_manager.cc:1428] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:22.953641 1645 catalog_manager.cc:4841] TS ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045): tablet d5f27006d0004d2a957b15be22e9d7a7 (table test-table [id=be6f3644df2342c2b3fb65d5f7e1db22]) successfully deleted
I20260810 18:53:22.955579 2466 tablet_bootstrap.cc:492] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: No bootstrap required, opened a new log
I20260810 18:53:22.955690 2466 ts_tablet_manager.cc:1397] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260810 18:53:22.956233 2466 raft_consensus.cc:348] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.956353 2466 raft_consensus.cc:374] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:22.956410 2466 raft_consensus.cc:729] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc6ff59ea6d4676bce6984b87e88a26, State: Initialized, Role: FOLLOWER
I20260810 18:53:22.956545 2466 consensus_queue.cc:260] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:22.956830 2466 ts_tablet_manager.cc:1428] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:53:23.131090 2476 raft_consensus.cc:482] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:23.131253 2476 raft_consensus.cc:504] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:23.132396 2476 leader_election.cc:288] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045), 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085)
I20260810 18:53:23.138262 2538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.132752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:48821 (local address 127.30.153.130:33045)
0810 18:53:23.132981 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:23.132986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:23.133100 (+ 114us) server_negotiation.cc:407] Connection header received
0810 18:53:23.133280 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:23.133283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:23.133343 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:23.133438 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:23.133903 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.134752 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.136774 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.137426 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.137506 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.137581 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:23.138047 (+ 466us) server_negotiation.cc:299] Negotiation successful
0810 18:53:23.138142 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
I20260810 18:53:23.138303 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.132739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:36085 (local address 127.30.153.129:56889)
0810 18:53:23.133114 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:23.133128 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:23.133220 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:23.133499 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:23.133503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:23.133544 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:23.133759 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.133767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.134876 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.134880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.136611 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.136622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.137630 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.137634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.137855 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.137893 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 18:53:23.138010 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 18:53:23.138098 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":260,"spinlock_wait_cycles":6912,"thread_start_us":127,"threads_started":1}
I20260810 18:53:23.138315 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.132618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:33045 (local address 127.30.153.129:48821)
0810 18:53:23.132980 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:23.132996 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:23.133141 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:23.133499 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:23.133503 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:23.133535 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:23.133759 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.133767 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.134921 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.134923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.136609 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.136622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.137675 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.137678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.137850 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.137896 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 18:53:23.138010 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 18:53:23.138098 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":268,"spinlock_wait_cycles":8832,"thread_start_us":106,"threads_started":1}
I20260810 18:53:23.138716 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.132831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:56889 (local address 127.30.153.131:36085)
0810 18:53:23.132984 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:23.132989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:23.133225 (+ 236us) server_negotiation.cc:407] Connection header received
0810 18:53:23.133301 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:23.133303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:23.133346 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:23.133437 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:23.133903 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.134727 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.136800 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.137476 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.137615 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.137667 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:23.138499 (+ 832us) server_negotiation.cc:299] Negotiation successful
0810 18:53:23.138597 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:23.139051 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b1fa843ad1e49f493b50f48d3fe0eab" candidate_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" is_pre_election: true
I20260810 18:53:23.139201 2150 raft_consensus.cc:2454] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc6ff59ea6d4676bce6984b87e88a26 in term 0.
I20260810 18:53:23.139200 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b1fa843ad1e49f493b50f48d3fe0eab" candidate_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bac1656e120481aa7e90bbd96d6fd51" is_pre_election: true
I20260810 18:53:23.139456 2366 raft_consensus.cc:2454] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc6ff59ea6d4676bce6984b87e88a26 in term 0.
I20260810 18:53:23.139622 1822 leader_election.cc:302] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfc6ff59ea6d4676bce6984b87e88a26, ee1e515f85bc4029bf78c0ab2aa7833a; no voters:
I20260810 18:53:23.139946 2476 raft_consensus.cc:2781] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:23.140034 2476 raft_consensus.cc:482] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:23.140105 2476 raft_consensus.cc:3037] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:23.141284 2476 raft_consensus.cc:504] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:23.141669 2476 leader_election.cc:288] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [CANDIDATE]: Term 1 election: Requested vote from peers ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045), 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085)
I20260810 18:53:23.142084 2150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b1fa843ad1e49f493b50f48d3fe0eab" candidate_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a"
I20260810 18:53:23.142094 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b1fa843ad1e49f493b50f48d3fe0eab" candidate_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bac1656e120481aa7e90bbd96d6fd51"
I20260810 18:53:23.142226 2150 raft_consensus.cc:3037] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:23.142226 2366 raft_consensus.cc:3037] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:23.143241 2150 raft_consensus.cc:2454] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc6ff59ea6d4676bce6984b87e88a26 in term 1.
I20260810 18:53:23.143244 2366 raft_consensus.cc:2454] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc6ff59ea6d4676bce6984b87e88a26 in term 1.
I20260810 18:53:23.143574 1822 leader_election.cc:302] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfc6ff59ea6d4676bce6984b87e88a26, ee1e515f85bc4029bf78c0ab2aa7833a; no voters:
I20260810 18:53:23.143790 2476 raft_consensus.cc:2781] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:23.143980 2476 raft_consensus.cc:686] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 LEADER]: Becoming Leader. State: Replica: dfc6ff59ea6d4676bce6984b87e88a26, State: Running, Role: LEADER
I20260810 18:53:23.144291 2476 consensus_queue.cc:237] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:23.146827 1665 catalog_manager.cc:5495] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129). New cstate: current_term: 1 leader_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:23.197017 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.193135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.30.153.129:34089)
0810 18:53:23.193420 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:23.193439 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:23.193453 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:53:23.193532 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:23.193534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:23.193584 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:23.193719 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:23.193970 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.194594 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.195459 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.195702 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.195813 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.196091 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:23.196552 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:23.196693 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:23.196755 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 18:53:23.196832 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":105,"threads_started":1}
I20260810 18:53:23.208384 2458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.205012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:59777 (local address 127.30.153.131:36085)
0810 18:53:23.205221 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:23.205226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:23.205332 (+ 106us) server_negotiation.cc:407] Connection header received
0810 18:53:23.205432 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:23.205434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:23.205483 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:23.205560 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:23.205823 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.206360 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.207309 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.207526 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.207617 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.207879 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:23.207947 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:23.208000 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:23.208222 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 18:53:23.208266 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260810 18:53:23.208398 2548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.204913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:36085 (local address 127.30.153.129:59777)
0810 18:53:23.205246 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:23.205259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:23.205346 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:23.205578 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:23.205580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:23.205586 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:23.205697 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.205701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.206573 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.206575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.207205 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.207214 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.207645 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.207648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.207719 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.207760 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:23.208122 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:23.208130 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:53:23.208199 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 18:53:23.208267 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":127,"threads_started":1}
I20260810 18:53:23.215090 2150 raft_consensus.cc:1261] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Refusing update from remote peer dfc6ff59ea6d4676bce6984b87e88a26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:23.215323 2366 raft_consensus.cc:1261] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Refusing update from remote peer dfc6ff59ea6d4676bce6984b87e88a26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:23.215703 2476 consensus_queue.cc:1035] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:23.215912 2541 consensus_queue.cc:1035] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:23.234243 2521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.230985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:34089 (local address 127.30.153.131:50629)
0810 18:53:23.231147 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:23.231165 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:23.231256 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:23.231512 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:23.231514 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:23.231521 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:23.231664 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.231671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.232439 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.232442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.233113 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:23.233123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.233596 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.233599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:23.233661 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.233688 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:23.233943 (+ 255us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:23.233953 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:53:23.234035 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:53:23.234125 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 18:53:23.234243 2543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:23.231036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:50629 (local address 127.30.153.129:34089)
0810 18:53:23.231145 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:23.231149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:23.231210 (+ 61us) server_negotiation.cc:407] Connection header received
0810 18:53:23.231356 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:23.231359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:23.231408 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:23.231483 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:23.231786 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.232316 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.233239 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:23.233484 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:23.233575 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:23.233798 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:23.233865 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:23.233922 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:23.234053 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 18:53:23.234109 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 18:53:23.235314 2506 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 18:53:24.809226 1665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:24.809381 1665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:24.812776 1665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: Sending DeleteTablet for 3 replicas of tablet 5b1fa843ad1e49f493b50f48d3fe0eab
I20260810 18:53:24.813359 1896 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b1fa843ad1e49f493b50f48d3fe0eab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:24 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:40924
I20260810 18:53:24.813517 2120 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b1fa843ad1e49f493b50f48d3fe0eab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:24 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:35604
I20260810 18:53:24.813591 2336 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b1fa843ad1e49f493b50f48d3fe0eab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:24 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33156
I20260810 18:53:24.813927 2586 tablet_replica.cc:330] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: stopping tablet replica
I20260810 18:53:24.814044 2587 tablet_replica.cc:330] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: stopping tablet replica
I20260810 18:53:24.814159 2586 raft_consensus.cc:2229] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:24.814270 2587 raft_consensus.cc:2229] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:24.814368 2586 raft_consensus.cc:2258] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:24.814466 2587 raft_consensus.cc:2258] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:24.815187 2586 ts_tablet_manager.cc:1905] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:24.815403 2587 ts_tablet_manager.cc:1905] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:24.817428 2585 tablet_replica.cc:330] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: stopping tablet replica
I20260810 18:53:24.817637 2585 raft_consensus.cc:2229] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:24.818023 2585 raft_consensus.cc:2258] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:24.818440 2586 ts_tablet_manager.cc:1918] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:53:24.818531 2586 log.cc:1196] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/5b1fa843ad1e49f493b50f48d3fe0eab
I20260810 18:53:24.818758 2587 ts_tablet_manager.cc:1918] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:53:24.818830 2586 ts_tablet_manager.cc:1939] T 5b1fa843ad1e49f493b50f48d3fe0eab P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting consensus metadata
I20260810 18:53:24.818852 2587 log.cc:1196] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/5b1fa843ad1e49f493b50f48d3fe0eab
I20260810 18:53:24.819111 2587 ts_tablet_manager.cc:1939] T 5b1fa843ad1e49f493b50f48d3fe0eab P 1bac1656e120481aa7e90bbd96d6fd51: Deleting consensus metadata
I20260810 18:53:24.819613 1645 catalog_manager.cc:4841] TS ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045): tablet 5b1fa843ad1e49f493b50f48d3fe0eab (table test-table [id=6a1349adc0db49d9a6e21f8166734b91]) successfully deleted
I20260810 18:53:24.819746 1644 catalog_manager.cc:4841] TS 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085): tablet 5b1fa843ad1e49f493b50f48d3fe0eab (table test-table [id=6a1349adc0db49d9a6e21f8166734b91]) successfully deleted
I20260810 18:53:24.820926 2585 ts_tablet_manager.cc:1905] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:24.823119 2585 ts_tablet_manager.cc:1918] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:53:24.823186 2585 log.cc:1196] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/5b1fa843ad1e49f493b50f48d3fe0eab
I20260810 18:53:24.823487 2585 ts_tablet_manager.cc:1939] T 5b1fa843ad1e49f493b50f48d3fe0eab P dfc6ff59ea6d4676bce6984b87e88a26: Deleting consensus metadata
I20260810 18:53:24.824225 1645 catalog_manager.cc:4841] TS dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089): tablet 5b1fa843ad1e49f493b50f48d3fe0eab (table test-table [id=6a1349adc0db49d9a6e21f8166734b91]) successfully deleted
Aug 10 18:53:24 dist-test-slave-68xk krb5kdc[1599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:53:24 dist-test-slave-68xk krb5kdc[1599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:24.853214 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.837014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.30.153.190:40917)
0810 18:53:24.837343 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.837351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.837412 (+ 61us) server_negotiation.cc:407] Connection header received
0810 18:53:24.837544 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.837547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.837589 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.837672 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:24.838606 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.839227 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.840106 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.840374 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.840484 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.849078 (+ 8594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:24.849082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:24.849085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:24.849127 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:24.851068 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:24.851685 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:24.851690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:24.851692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:24.851791 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:24.852126 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:24.852129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:24.852131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:24.852355 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:24.852544 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.852820 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.852974 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":114,"threads_started":1}
I20260810 18:53:24.856755 1665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:24.857263 1665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:24.862673 1896 tablet_service.cc:1467] Processing CreateTablet for tablet cabe18e690184e89b343643a09759f25 (DEFAULT_TABLE table=test-table [id=e6196c6bdf6e40ef8de4d54a5bb52443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:24.862694 2120 tablet_service.cc:1467] Processing CreateTablet for tablet cabe18e690184e89b343643a09759f25 (DEFAULT_TABLE table=test-table [id=e6196c6bdf6e40ef8de4d54a5bb52443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:24.862787 2336 tablet_service.cc:1467] Processing CreateTablet for tablet cabe18e690184e89b343643a09759f25 (DEFAULT_TABLE table=test-table [id=e6196c6bdf6e40ef8de4d54a5bb52443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:24.863070 2120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cabe18e690184e89b343643a09759f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:24.863096 1896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cabe18e690184e89b343643a09759f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:24.863162 2336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cabe18e690184e89b343643a09759f25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:24.866092 2602 tablet_bootstrap.cc:492] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: Bootstrap starting.
I20260810 18:53:24.866243 2609 tablet_bootstrap.cc:492] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: Bootstrap starting.
I20260810 18:53:24.866294 2608 tablet_bootstrap.cc:492] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: Bootstrap starting.
I20260810 18:53:24.867245 2609 tablet_bootstrap.cc:654] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:24.867245 2608 tablet_bootstrap.cc:654] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:24.867501 2602 tablet_bootstrap.cc:654] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:24.868637 2609 tablet_bootstrap.cc:492] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: No bootstrap required, opened a new log
I20260810 18:53:24.868798 2609 ts_tablet_manager.cc:1397] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:24.868844 2608 tablet_bootstrap.cc:492] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: No bootstrap required, opened a new log
I20260810 18:53:24.869050 2608 ts_tablet_manager.cc:1397] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:24.869103 2602 tablet_bootstrap.cc:492] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: No bootstrap required, opened a new log
I20260810 18:53:24.869189 2602 ts_tablet_manager.cc:1397] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:24.869351 2609 raft_consensus.cc:348] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.869480 2609 raft_consensus.cc:374] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:24.869519 2609 raft_consensus.cc:729] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bac1656e120481aa7e90bbd96d6fd51, State: Initialized, Role: FOLLOWER
I20260810 18:53:24.869520 2608 raft_consensus.cc:348] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.869652 2608 raft_consensus.cc:374] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:24.869702 2608 raft_consensus.cc:729] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc6ff59ea6d4676bce6984b87e88a26, State: Initialized, Role: FOLLOWER
I20260810 18:53:24.869700 2602 raft_consensus.cc:348] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.869834 2602 raft_consensus.cc:374] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:24.869881 2602 raft_consensus.cc:729] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1e515f85bc4029bf78c0ab2aa7833a, State: Initialized, Role: FOLLOWER
I20260810 18:53:24.869868 2608 consensus_queue.cc:260] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.870023 2602 consensus_queue.cc:260] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.870589 2602 ts_tablet_manager.cc:1428] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260810 18:53:24.871014 2609 consensus_queue.cc:260] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.871389 2609 ts_tablet_manager.cc:1428] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:24.871994 2608 ts_tablet_manager.cc:1428] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260810 18:53:24.982712 2613 raft_consensus.cc:482] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:24.982921 2613 raft_consensus.cc:504] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.984090 2613 leader_election.cc:288] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089), 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085)
I20260810 18:53:24.988474 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.984454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:41429 (local address 127.30.153.129:34089)
0810 18:53:24.984840 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.984850 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.984954 (+ 104us) server_negotiation.cc:407] Connection header received
0810 18:53:24.985286 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.985289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.985332 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.985436 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:24.985722 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.986345 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.987462 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.987894 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.988026 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.988083 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.988270 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.988341 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":102,"threads_started":1}
I20260810 18:53:24.988657 2618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.984500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:43097 (local address 127.30.153.131:36085)
0810 18:53:24.985252 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:24.985262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:24.985278 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:24.985333 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:24.985335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:24.985381 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:24.985466 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:24.985908 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.986573 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.987666 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.988068 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.988204 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.988251 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:24.988440 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 18:53:24.988511 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":99,"threads_started":1}
I20260810 18:53:24.988678 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.984345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:36085 (local address 127.30.153.130:43097)
0810 18:53:24.984753 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.984770 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:24.984955 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:24.985619 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:24.985623 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:24.985643 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:24.985773 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.985781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.986747 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.986750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.987530 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.987539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.988197 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.988200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.988299 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.988332 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 18:53:24.988421 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 18:53:24.988490 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":22,"thread_start_us":146,"threads_started":1}
I20260810 18:53:24.988487 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:24.984349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:34089 (local address 127.30.153.130:41429)
0810 18:53:24.984836 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:24.984854 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:24.984981 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:24.985448 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:24.985451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:24.985467 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:24.985600 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.985607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.986494 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.986498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.987339 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:24.987348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:24.988021 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:24.988025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:24.988144 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:24.988163 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 18:53:24.988252 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 18:53:24.988325 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":54,"thread_start_us":131,"threads_started":1}
I20260810 18:53:24.989168 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cabe18e690184e89b343643a09759f25" candidate_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bac1656e120481aa7e90bbd96d6fd51" is_pre_election: true
I20260810 18:53:24.989321 2366 raft_consensus.cc:2454] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1e515f85bc4029bf78c0ab2aa7833a in term 0.
I20260810 18:53:24.989300 1931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cabe18e690184e89b343643a09759f25" candidate_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" is_pre_election: true
I20260810 18:53:24.989460 1931 raft_consensus.cc:2454] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1e515f85bc4029bf78c0ab2aa7833a in term 0.
I20260810 18:53:24.989809 2049 leader_election.cc:302] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bac1656e120481aa7e90bbd96d6fd51, ee1e515f85bc4029bf78c0ab2aa7833a; no voters:
I20260810 18:53:24.990077 2613 raft_consensus.cc:2781] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:24.990187 2613 raft_consensus.cc:482] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:24.990244 2613 raft_consensus.cc:3037] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:24.991339 2613 raft_consensus.cc:504] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.991726 2613 leader_election.cc:288] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [CANDIDATE]: Term 1 election: Requested vote from peers dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089), 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085)
I20260810 18:53:24.992161 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cabe18e690184e89b343643a09759f25" candidate_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bac1656e120481aa7e90bbd96d6fd51"
I20260810 18:53:24.992166 1931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cabe18e690184e89b343643a09759f25" candidate_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc6ff59ea6d4676bce6984b87e88a26"
I20260810 18:53:24.992287 1931 raft_consensus.cc:3037] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:24.992287 2366 raft_consensus.cc:3037] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:24.993427 1931 raft_consensus.cc:2454] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1e515f85bc4029bf78c0ab2aa7833a in term 1.
I20260810 18:53:24.993427 2366 raft_consensus.cc:2454] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1e515f85bc4029bf78c0ab2aa7833a in term 1.
I20260810 18:53:24.993716 2049 leader_election.cc:302] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bac1656e120481aa7e90bbd96d6fd51, ee1e515f85bc4029bf78c0ab2aa7833a; no voters:
I20260810 18:53:24.993883 2613 raft_consensus.cc:2781] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:24.994129 2613 raft_consensus.cc:686] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 LEADER]: Becoming Leader. State: Replica: ee1e515f85bc4029bf78c0ab2aa7833a, State: Running, Role: LEADER
I20260810 18:53:24.994385 2613 consensus_queue.cc:237] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:24.996773 1663 catalog_manager.cc:5495] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a reported cstate change: term changed from 0 to 1, leader changed from <none> to ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130). New cstate: current_term: 1 leader_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:25.028515 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:25.024862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35618 (local address 127.30.153.130:33045)
0810 18:53:25.025164 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:25.025171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:25.025183 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:53:25.025257 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:25.025259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:25.025310 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:25.025414 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:25.025651 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.026220 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.027083 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.027359 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.027441 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:25.027715 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:25.028186 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:25.028284 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:25.028341 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 18:53:25.028388 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":138,"threads_started":1}
I20260810 18:53:25.039502 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:25.035712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:36085 (local address 127.30.153.130:53371)
0810 18:53:25.036187 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:25.036206 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:25.036293 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:25.036558 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:25.036563 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:25.036574 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:25.036686 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:25.036691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.037665 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.037667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:25.038278 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:25.038286 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.038866 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.038869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:25.038941 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:25.038984 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:25.039266 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:25.039271 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:53:25.039319 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 18:53:25.039383 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":109,"threads_started":1}
I20260810 18:53:25.039505 2618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:25.035906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:53371 (local address 127.30.153.131:36085)
0810 18:53:25.036034 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:25.036039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:25.036277 (+ 238us) server_negotiation.cc:407] Connection header received
0810 18:53:25.036364 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:25.036367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:25.036422 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:25.036502 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:25.036778 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.037531 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.038389 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.038766 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.038863 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:25.039111 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:25.039187 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:25.039256 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:25.039330 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 18:53:25.039380 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:25.045851 2366 raft_consensus.cc:1261] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Refusing update from remote peer ee1e515f85bc4029bf78c0ab2aa7833a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:25.045966 1931 raft_consensus.cc:1261] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Refusing update from remote peer ee1e515f85bc4029bf78c0ab2aa7833a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:25.046458 2619 consensus_queue.cc:1035] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:25.046667 2613 consensus_queue.cc:1035] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:25.064188 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:25.060625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:58173 (local address 127.30.153.130:33045)
0810 18:53:25.060760 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:25.060765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:25.060919 (+ 154us) server_negotiation.cc:407] Connection header received
0810 18:53:25.061049 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:25.061052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:25.061096 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:25.061188 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:25.061466 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.062101 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.063203 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.063549 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.063644 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:25.063847 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:25.063909 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:25.063961 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:25.064014 (+ 53us) server_negotiation.cc:299] Negotiation successful
0810 18:53:25.064054 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:53:25.064184 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:25.060513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:33045 (local address 127.30.153.131:58173)
0810 18:53:25.060821 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:25.060835 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:25.060961 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:25.061196 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:25.061199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:25.061207 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:25.061341 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:25.061347 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.062207 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.062210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:25.063084 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:25.063093 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:25.063649 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:25.063652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:25.063716 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:25.063743 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:25.063961 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:25.063964 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 18:53:25.064007 (+ 43us) client_negotiation.cc:240] Negotiation successful
0810 18:53:25.064070 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":118,"threads_started":1}
I20260810 18:53:25.085404 1665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:25.085614 1665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:25.087800 1665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: Sending DeleteTablet for 3 replicas of tablet cabe18e690184e89b343643a09759f25
I20260810 18:53:25.088362 1896 tablet_service.cc:1514] Processing DeleteTablet for tablet cabe18e690184e89b343643a09759f25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:25 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:40924
I20260810 18:53:25.088459 2120 tablet_service.cc:1514] Processing DeleteTablet for tablet cabe18e690184e89b343643a09759f25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:25 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:35604
I20260810 18:53:25.088583 2585 tablet_replica.cc:330] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: stopping tablet replica
I20260810 18:53:25.088642 2586 tablet_replica.cc:330] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: stopping tablet replica
I20260810 18:53:25.088727 2585 raft_consensus.cc:2229] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:25.088762 2586 raft_consensus.cc:2229] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:25.088868 2585 raft_consensus.cc:2258] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:25.089027 2586 raft_consensus.cc:2258] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:25.089370 2336 tablet_service.cc:1514] Processing DeleteTablet for tablet cabe18e690184e89b343643a09759f25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:25 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33156
I20260810 18:53:25.089555 2587 tablet_replica.cc:330] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: stopping tablet replica
I20260810 18:53:25.089672 2587 raft_consensus.cc:2229] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:25.089730 2586 ts_tablet_manager.cc:1905] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:25.089772 2587 raft_consensus.cc:2258] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:25.089948 2585 ts_tablet_manager.cc:1905] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:25.090492 2587 ts_tablet_manager.cc:1905] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:25.092639 2586 ts_tablet_manager.cc:1918] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:25.092676 2585 ts_tablet_manager.cc:1918] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:25.092762 2585 log.cc:1196] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/cabe18e690184e89b343643a09759f25
I20260810 18:53:25.092877 2586 log.cc:1196] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/cabe18e690184e89b343643a09759f25
I20260810 18:53:25.093045 2585 ts_tablet_manager.cc:1939] T cabe18e690184e89b343643a09759f25 P dfc6ff59ea6d4676bce6984b87e88a26: Deleting consensus metadata
I20260810 18:53:25.093161 2587 ts_tablet_manager.cc:1918] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:25.093246 2587 log.cc:1196] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/cabe18e690184e89b343643a09759f25
I20260810 18:53:25.093431 2586 ts_tablet_manager.cc:1939] T cabe18e690184e89b343643a09759f25 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting consensus metadata
I20260810 18:53:25.093513 2587 ts_tablet_manager.cc:1939] T cabe18e690184e89b343643a09759f25 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting consensus metadata
I20260810 18:53:25.093873 1645 catalog_manager.cc:4841] TS dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089): tablet cabe18e690184e89b343643a09759f25 (table test-table [id=e6196c6bdf6e40ef8de4d54a5bb52443]) successfully deleted
I20260810 18:53:25.094264 1644 catalog_manager.cc:4841] TS 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085): tablet cabe18e690184e89b343643a09759f25 (table test-table [id=e6196c6bdf6e40ef8de4d54a5bb52443]) successfully deleted
I20260810 18:53:25.094671 1645 catalog_manager.cc:4841] TS ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045): tablet cabe18e690184e89b343643a09759f25 (table test-table [id=e6196c6bdf6e40ef8de4d54a5bb52443]) successfully deleted
I20260810 18:53:25.094871 1665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:25.095302 1665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:25.100061 1896 tablet_service.cc:1467] Processing CreateTablet for tablet eeb9c26283254e098e98503eb9a96825 (DEFAULT_TABLE table=test-table [id=d5b7f8af971a49d9998565d5bd607650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:25.100102 2120 tablet_service.cc:1467] Processing CreateTablet for tablet eeb9c26283254e098e98503eb9a96825 (DEFAULT_TABLE table=test-table [id=d5b7f8af971a49d9998565d5bd607650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:25.100361 2120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb9c26283254e098e98503eb9a96825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:25.100365 1896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb9c26283254e098e98503eb9a96825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:25.102161 2608 tablet_bootstrap.cc:492] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26: Bootstrap starting.
I20260810 18:53:25.102516 2336 tablet_service.cc:1467] Processing CreateTablet for tablet eeb9c26283254e098e98503eb9a96825 (DEFAULT_TABLE table=test-table [id=d5b7f8af971a49d9998565d5bd607650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:25.102623 2602 tablet_bootstrap.cc:492] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: Bootstrap starting.
I20260810 18:53:25.102777 2336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eeb9c26283254e098e98503eb9a96825. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:25.103070 2608 tablet_bootstrap.cc:654] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:25.103581 2602 tablet_bootstrap.cc:654] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:25.104899 2602 tablet_bootstrap.cc:492] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: No bootstrap required, opened a new log
I20260810 18:53:25.104986 2602 ts_tablet_manager.cc:1397] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260810 18:53:25.105419 2602 raft_consensus.cc:348] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.105531 2602 raft_consensus.cc:374] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:25.105563 2602 raft_consensus.cc:729] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1e515f85bc4029bf78c0ab2aa7833a, State: Initialized, Role: FOLLOWER
I20260810 18:53:25.105681 2602 consensus_queue.cc:260] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.105949 2602 ts_tablet_manager.cc:1428] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260810 18:53:25.106137 2608 tablet_bootstrap.cc:492] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26: No bootstrap required, opened a new log
I20260810 18:53:25.106138 2609 tablet_bootstrap.cc:492] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: Bootstrap starting.
I20260810 18:53:25.106197 2608 ts_tablet_manager.cc:1397] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260810 18:53:25.106575 2608 raft_consensus.cc:348] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.106698 2608 raft_consensus.cc:374] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:25.106741 2608 raft_consensus.cc:729] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc6ff59ea6d4676bce6984b87e88a26, State: Initialized, Role: FOLLOWER
I20260810 18:53:25.106910 2609 tablet_bootstrap.cc:654] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:25.106876 2608 consensus_queue.cc:260] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.107144 2608 ts_tablet_manager.cc:1428] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:25.108083 2609 tablet_bootstrap.cc:492] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: No bootstrap required, opened a new log
I20260810 18:53:25.108170 2609 ts_tablet_manager.cc:1397] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:25.108566 2609 raft_consensus.cc:348] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.108649 2609 raft_consensus.cc:374] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:25.108703 2609 raft_consensus.cc:729] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bac1656e120481aa7e90bbd96d6fd51, State: Initialized, Role: FOLLOWER
I20260810 18:53:25.108843 2609 consensus_queue.cc:260] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.109122 2609 ts_tablet_manager.cc:1428] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:25.132001 2640 raft_consensus.cc:482] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:25.132180 2640 raft_consensus.cc:504] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.132709 2640 leader_election.cc:288] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089), 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085)
I20260810 18:53:25.133154 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9c26283254e098e98503eb9a96825" candidate_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bac1656e120481aa7e90bbd96d6fd51" is_pre_election: true
I20260810 18:53:25.133180 1931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9c26283254e098e98503eb9a96825" candidate_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" is_pre_election: true
I20260810 18:53:25.133320 2366 raft_consensus.cc:2454] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1e515f85bc4029bf78c0ab2aa7833a in term 0.
I20260810 18:53:25.133343 1931 raft_consensus.cc:2454] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee1e515f85bc4029bf78c0ab2aa7833a in term 0.
I20260810 18:53:25.133620 2049 leader_election.cc:302] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bac1656e120481aa7e90bbd96d6fd51, ee1e515f85bc4029bf78c0ab2aa7833a; no voters:
I20260810 18:53:25.133818 2640 raft_consensus.cc:2781] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:25.133886 2640 raft_consensus.cc:482] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:25.133925 2640 raft_consensus.cc:3037] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:25.136122 2640 raft_consensus.cc:504] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.136686 2640 leader_election.cc:288] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [CANDIDATE]: Term 1 election: Requested vote from peers dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089), 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085)
I20260810 18:53:25.137102 2366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9c26283254e098e98503eb9a96825" candidate_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bac1656e120481aa7e90bbd96d6fd51"
I20260810 18:53:25.137108 1931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eeb9c26283254e098e98503eb9a96825" candidate_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc6ff59ea6d4676bce6984b87e88a26"
I20260810 18:53:25.137225 2366 raft_consensus.cc:3037] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:25.137236 1931 raft_consensus.cc:3037] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:25.138393 1931 raft_consensus.cc:2454] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1e515f85bc4029bf78c0ab2aa7833a in term 1.
I20260810 18:53:25.138648 2366 raft_consensus.cc:2454] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee1e515f85bc4029bf78c0ab2aa7833a in term 1.
I20260810 18:53:25.138772 2050 leader_election.cc:302] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dfc6ff59ea6d4676bce6984b87e88a26, ee1e515f85bc4029bf78c0ab2aa7833a; no voters:
I20260810 18:53:25.138948 2640 raft_consensus.cc:2781] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:25.139120 2640 raft_consensus.cc:686] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 LEADER]: Becoming Leader. State: Replica: ee1e515f85bc4029bf78c0ab2aa7833a, State: Running, Role: LEADER
I20260810 18:53:25.139268 2640 consensus_queue.cc:237] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } }
I20260810 18:53:25.140961 1666 catalog_manager.cc:5495] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a reported cstate change: term changed from 0 to 1, leader changed from <none> to ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130). New cstate: current_term: 1 leader_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee1e515f85bc4029bf78c0ab2aa7833a" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:25.162551 1931 raft_consensus.cc:1261] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26 [term 1 FOLLOWER]: Refusing update from remote peer ee1e515f85bc4029bf78c0ab2aa7833a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:25.162541 2366 raft_consensus.cc:1261] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Refusing update from remote peer ee1e515f85bc4029bf78c0ab2aa7833a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:25.163023 2640 consensus_queue.cc:1035] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bac1656e120481aa7e90bbd96d6fd51" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:25.163189 2613 consensus_queue.cc:1035] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfc6ff59ea6d4676bce6984b87e88a26" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 34089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:53:25.178256 2559 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 18:53:26.731999 1666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:26.732213 1666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:26.734889 1666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9acbf5911d834a119807578a5e7dee04: Sending DeleteTablet for 3 replicas of tablet eeb9c26283254e098e98503eb9a96825
I20260810 18:53:26.735411 1896 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb9c26283254e098e98503eb9a96825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:26 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:40924
I20260810 18:53:26.735467 2120 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb9c26283254e098e98503eb9a96825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:26 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:35604
I20260810 18:53:26.735781 2687 tablet_replica.cc:330] T eeb9c26283254e098e98503eb9a96825 P dfc6ff59ea6d4676bce6984b87e88a26: stopping tablet replica
I20260810 18:53:26.735769 2336 tablet_service.cc:1514] Processing DeleteTablet for tablet eeb9c26283254e098e98503eb9a96825 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:26 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33156
I20260810 18:53:26.735812 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 1787
I20260810 18:53:26.745048 2688 tablet_replica.cc:330] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: stopping tablet replica
I20260810 18:53:26.745260 2688 raft_consensus.cc:2229] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:26.745532 2688 raft_consensus.cc:2258] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:26.746464 2688 ts_tablet_manager.cc:1905] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:26.748958 2688 ts_tablet_manager.cc:1918] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:53:26.749032 2688 log.cc:1196] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/eeb9c26283254e098e98503eb9a96825
I20260810 18:53:26.749325 2688 ts_tablet_manager.cc:1939] T eeb9c26283254e098e98503eb9a96825 P ee1e515f85bc4029bf78c0ab2aa7833a: Deleting consensus metadata
I20260810 18:53:26.750072 1645 catalog_manager.cc:4841] TS ee1e515f85bc4029bf78c0ab2aa7833a (127.30.153.130:33045): tablet eeb9c26283254e098e98503eb9a96825 (table test-table [id=d5b7f8af971a49d9998565d5bd607650]) successfully deleted
I20260810 18:53:26.750675 2689 tablet_replica.cc:330] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: stopping tablet replica
I20260810 18:53:26.750860 2689 raft_consensus.cc:2229] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:26.751009 2689 raft_consensus.cc:2258] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:26.751636 2689 ts_tablet_manager.cc:1905] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:26.753795 2689 ts_tablet_manager.cc:1918] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 18:53:26.753891 2689 log.cc:1196] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/eeb9c26283254e098e98503eb9a96825
I20260810 18:53:26.754153 2689 ts_tablet_manager.cc:1939] T eeb9c26283254e098e98503eb9a96825 P 1bac1656e120481aa7e90bbd96d6fd51: Deleting consensus metadata
I20260810 18:53:26.755172 1644 catalog_manager.cc:4841] TS 1bac1656e120481aa7e90bbd96d6fd51 (127.30.153.131:36085): tablet eeb9c26283254e098e98503eb9a96825 (table test-table [id=d5b7f8af971a49d9998565d5bd607650]) successfully deleted
I20260810 18:53:26.755267 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 2024
W20260810 18:53:26.763433 1645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:34089 (error 108)
I20260810 18:53:26.764845 2692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:26.764418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:34089 (local address 127.0.0.1:40934)
0810 18:53:26.764670 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:26.764754 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":105,"threads_started":1}
W20260810 18:53:26.764986 1645 catalog_manager.cc:4584] TS dfc6ff59ea6d4676bce6984b87e88a26 (127.30.153.129:34089): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eeb9c26283254e098e98503eb9a96825: Network error: Client connection negotiation failed: client connection to 127.30.153.129:34089: connect: Connection refused (error 111)
I20260810 18:53:26.770907 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 2249
I20260810 18:53:26.781831 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 1619
2026-08-10T18:53:26Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6948 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:26 dist-test-slave-68xk krb5kdc[2695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:26 dist-test-slave-68xk krb5kdc[2695](info): set up 2 sockets
Aug 10 18:53:26 dist-test-slave-68xk krb5kdc[2695](info): commencing operation
krb5kdc: starting...
W20260810 18:53:28.041096 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:28 dist-test-slave-68xk krb5kdc[2695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:28.177815 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:45223
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:45223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:28.292538 2715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:28.292816 2715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:28.292858 2715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:28.296906 2715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:28.296959 2715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:28.296978 2715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:28.296995 2715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:28.297010 2715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:28.302310 2715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:45223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:45223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:28.303354 2715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:28.304385 2715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:28.314013 2726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:28.314118 2722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:28.314150 2729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:28.314273 2715 server_base.cc:1034] running on GCE node
I20260810 18:53:28.314949 2715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:28.316116 2715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:28.317333 2715 hybrid_clock.cc:648] HybridClock initialized: now 1786388008317309 us; error 49 us; skew 500 ppm
Aug 10 18:53:28 dist-test-slave-68xk krb5kdc[2695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:28.320360 2715 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:28.321367 2715 webserver.cc:460] Webserver started at http://127.30.153.190:39989/ using document root <none> and password file <none>
I20260810 18:53:28.321972 2715 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:28.322077 2715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:28.322335 2715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:28.324509 2715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "c0b60264555b45d5914f59c7c8a712f9"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "b49e1beb710fd24b65918d96b50640cf"
server_key_iv: "573515da94e9d766f84f0068fa45d38a"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.325140 2715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "c0b60264555b45d5914f59c7c8a712f9"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "b49e1beb710fd24b65918d96b50640cf"
server_key_iv: "573515da94e9d766f84f0068fa45d38a"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.329232 2715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 18:53:28.331921 2739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.333299 2715 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:53:28.333478 2715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "c0b60264555b45d5914f59c7c8a712f9"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "b49e1beb710fd24b65918d96b50640cf"
server_key_iv: "573515da94e9d766f84f0068fa45d38a"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.333611 2715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:28.351306 2715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:28.355145 2715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:28.355391 2715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:28.363616 2715 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:45223
I20260810 18:53:28.363665 2814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:45223 every 8 connection(s)
I20260810 18:53:28.364692 2715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:28.365113 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 2715
I20260810 18:53:28.365252 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:28.365522 31334 external_mini_cluster.cc:1428] Setting key 9eb431c15b25f8614fbba7bc9f2c6ae5
I20260810 18:53:28.368815 2815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 18:53:28 dist-test-slave-68xk krb5kdc[2695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:28.374958 2815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9: Bootstrap starting.
I20260810 18:53:28.377542 2815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:28.378362 2815 log.cc:824] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:28.379690 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.366881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.30.153.190:45223)
0810 18:53:28.367333 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:28.367348 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:28.367385 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:53:28.368032 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:28.368050 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:28.368406 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:28.368752 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:28.369563 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.370633 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.371501 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.371977 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.372167 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.374764 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:28.374769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:28.374782 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:28.374809 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:28.376628 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.377104 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.377115 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.377120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.377231 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.377550 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.377554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.377556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.377992 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:28.378206 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.378498 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.378782 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":134,"threads_started":1}
I20260810 18:53:28.380810 2815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9: No bootstrap required, opened a new log
I20260810 18:53:28.385383 2815 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b60264555b45d5914f59c7c8a712f9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 45223 } }
I20260810 18:53:28.385622 2815 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:28.385663 2815 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b60264555b45d5914f59c7c8a712f9, State: Initialized, Role: FOLLOWER
I20260810 18:53:28.386034 2815 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b60264555b45d5914f59c7c8a712f9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 45223 } }
I20260810 18:53:28.386164 2815 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:28.386219 2815 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:28.386296 2815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:28.387286 2815 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b60264555b45d5914f59c7c8a712f9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 45223 } }
I20260810 18:53:28.387614 2815 leader_election.cc:302] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0b60264555b45d5914f59c7c8a712f9; no voters:
I20260810 18:53:28.387912 2815 leader_election.cc:288] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:28.388048 2825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:28.388353 2825 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [term 1 LEADER]: Becoming Leader. State: Replica: c0b60264555b45d5914f59c7c8a712f9, State: Running, Role: LEADER
I20260810 18:53:28.388743 2825 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b60264555b45d5914f59c7c8a712f9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 45223 } }
I20260810 18:53:28.389271 2815 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:28.390890 2826 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0b60264555b45d5914f59c7c8a712f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b60264555b45d5914f59c7c8a712f9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 45223 } } }
I20260810 18:53:28.391023 2826 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:28.391396 2827 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0b60264555b45d5914f59c7c8a712f9. Latest consensus state: current_term: 1 leader_uuid: "c0b60264555b45d5914f59c7c8a712f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b60264555b45d5914f59c7c8a712f9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 45223 } } }
I20260810 18:53:28.391513 2827 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:28.391662 2840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:28.394726 2840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:28.400708 2840 catalog_manager.cc:1347] Generated new cluster ID: 71294b9f7be7480197e7d79bf944a83b
I20260810 18:53:28.400785 2840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:28.413100 2840 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:28.414240 2840 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:28.418941 2840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0b60264555b45d5914f59c7c8a712f9: Generated new TSK 0
I20260810 18:53:28.419803 2840 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:28.458848 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:45223
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:28.575976 2855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:28.576232 2855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:28.576275 2855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:28.580320 2855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:28.580379 2855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:28.580470 2855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:28.585726 2855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:28.586808 2855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:28.587845 2855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:28.598083 2868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:28.598172 2863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:28.598264 2855 server_base.cc:1034] running on GCE node
W20260810 18:53:28.598104 2864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:28.598779 2855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:28.599517 2855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:28.600704 2855 hybrid_clock.cc:648] HybridClock initialized: now 1786388008600669 us; error 46 us; skew 500 ppm
Aug 10 18:53:28 dist-test-slave-68xk krb5kdc[2695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:28.603721 2855 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:28.604702 2855 webserver.cc:460] Webserver started at http://127.30.153.129:35545/ using document root <none> and password file <none>
I20260810 18:53:28.605307 2855 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:28.605410 2855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:28.605706 2855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:28.607888 2855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "899b0215a7ec4ed488bc5bf700c27c7d"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "81195619e58bfc3fd50d1bc0f7d39679"
server_key_iv: "f937d2312fd7d5cc2188878e5856abab"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.608507 2855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "899b0215a7ec4ed488bc5bf700c27c7d"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "81195619e58bfc3fd50d1bc0f7d39679"
server_key_iv: "f937d2312fd7d5cc2188878e5856abab"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.612445 2855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:28.615068 2877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.616410 2855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:28.616572 2855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "899b0215a7ec4ed488bc5bf700c27c7d"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "81195619e58bfc3fd50d1bc0f7d39679"
server_key_iv: "f937d2312fd7d5cc2188878e5856abab"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.616696 2855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:28.632632 2855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:28.636283 2855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:28.636503 2855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:28.637205 2855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:28.638163 2855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:28.638238 2855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.638315 2855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:28.638373 2855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.647976 2855 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:41191
I20260810 18:53:28.647989 3045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:41191 every 8 connection(s)
I20260810 18:53:28.648986 2855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:28.656664 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 2855
I20260810 18:53:28.656765 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:28.657032 31334 external_mini_cluster.cc:1428] Setting key ab337c33cfa1d615ff2731eaddf9bc53
Aug 10 18:53:28 dist-test-slave-68xk krb5kdc[2695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:28.663201 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.650826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:53579 (local address 127.30.153.190:45223)
0810 18:53:28.650999 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:28.651002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:28.651811 (+ 809us) server_negotiation.cc:407] Connection header received
0810 18:53:28.653084 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:28.653087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:28.653132 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:28.653206 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:28.654318 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.654929 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.656108 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.656333 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.656466 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.659338 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:28.659342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:28.659343 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:28.659367 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:28.661085 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.661589 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.661596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.661598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.661712 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.662049 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.662053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.662055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.662238 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:28.662360 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.662902 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.663032 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:53:28.664083 3048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.651184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:45223 (local address 127.30.153.129:53579)
0810 18:53:28.651651 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:28.651694 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:28.652895 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:28.653339 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:28.653345 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:28.653704 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:28.654178 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.654192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.655065 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.655068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.655986 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.655996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.656506 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.656509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.656656 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.657315 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:28.657336 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:28.659171 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:28.661223 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:28.661228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:28.661240 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:28.661489 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:28.661812 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:28.661814 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:28.661815 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:28.661940 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:28.662360 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:28.662368 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:28.662658 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 18:53:28.662860 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 18:53:28.663174 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":113,"threads_started":1}
I20260810 18:53:28.665338 3046 heartbeater.cc:347] Connected to a master server at 127.30.153.190:45223
I20260810 18:53:28.665580 3046 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:28.666055 3046 heartbeater.cc:511] Master 127.30.153.190:45223 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:53:28.667807 2762 ts_manager.cc:194] Registered new tserver with Master: 899b0215a7ec4ed488bc5bf700c27c7d (127.30.153.129:41191)
I20260810 18:53:28.669126 2762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:53579
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:28.687544 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:45223
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:28.801229 3052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:28.801484 3052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:28.801528 3052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:28.805481 3052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:28.805542 3052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:28.805600 3052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:28.810881 3052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:28.811863 3052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:28.813078 3052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:28.822824 3060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:28.822834 3064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:28.822933 3052 server_base.cc:1034] running on GCE node
W20260810 18:53:28.822844 3061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:28.823541 3052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:28.824091 3052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:28.825266 3052 hybrid_clock.cc:648] HybridClock initialized: now 1786388008825235 us; error 32 us; skew 500 ppm
Aug 10 18:53:28 dist-test-slave-68xk krb5kdc[2695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:28.828173 3052 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:28.829061 3052 webserver.cc:460] Webserver started at http://127.30.153.130:34069/ using document root <none> and password file <none>
I20260810 18:53:28.829672 3052 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:28.829736 3052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:28.829952 3052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:28.832113 3052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "87231c61fe624c4ca98f3e4f3b95f982"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "78cd56da76e0f30b5fc2fb850a6c8ede"
server_key_iv: "88d5633fbaaf889f4316745d48602365"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.832674 3052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "87231c61fe624c4ca98f3e4f3b95f982"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "78cd56da76e0f30b5fc2fb850a6c8ede"
server_key_iv: "88d5633fbaaf889f4316745d48602365"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.836854 3052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 18:53:28.839682 3075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.840987 3052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:53:28.841169 3052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "87231c61fe624c4ca98f3e4f3b95f982"
format_stamp: "Formatted at 2026-08-10 18:53:28 on dist-test-slave-68xk"
server_key: "78cd56da76e0f30b5fc2fb850a6c8ede"
server_key_iv: "88d5633fbaaf889f4316745d48602365"
server_key_version: "encryptionkey@0"
I20260810 18:53:28.841289 3052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:28.869035 3052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:28.872323 3052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:28.872545 3052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:28.873183 3052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:28.874054 3052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:28.874130 3052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.874200 3052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:28.874253 3052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:28.884191 3052 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:37045
I20260810 18:53:28.884228 3242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:37045 every 8 connection(s)
I20260810 18:53:28.885246 3052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:28.894974 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 3052
I20260810 18:53:28.895179 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:28.895496 31334 external_mini_cluster.cc:1428] Setting key 52e77cf05ccad92175e8d1af2046a4f4
Aug 10 18:53:28 dist-test-slave-68xk krb5kdc[2695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:28.900642 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.887248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:45129 (local address 127.30.153.190:45223)
0810 18:53:28.887438 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:28.887444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:28.888744 (+ 1300us) server_negotiation.cc:407] Connection header received
0810 18:53:28.889761 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:28.889765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:28.889814 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:28.889898 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:28.891447 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.892154 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.893318 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.893540 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.893623 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.896253 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:28.896256 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:28.896259 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:28.896290 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:28.898230 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.898812 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.898819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.898824 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.898925 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:28.899300 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:28.899307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:28.899309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:28.899575 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:28.899664 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:28.900377 (+ 713us) server_negotiation.cc:299] Negotiation successful
0810 18:53:28.900497 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:53:28.901587 3247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:28.887650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:45223 (local address 127.30.153.130:45129)
0810 18:53:28.888568 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:28.888626 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:28.889506 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:28.890103 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:28.890110 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:28.890673 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:28.891251 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.891267 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.892301 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.892304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.893202 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:28.893212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:28.893677 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:28.893679 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:28.893829 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:28.894500 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:28.894516 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:28.896089 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:28.898360 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:28.898365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:28.898377 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:28.898709 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:28.899043 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:28.899045 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:28.899046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:28.899193 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:28.899714 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:28.899724 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:28.900062 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 18:53:28.900282 (+ 220us) client_negotiation.cc:240] Negotiation successful
0810 18:53:28.900672 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":546,"threads_started":1}
I20260810 18:53:28.902844 3243 heartbeater.cc:347] Connected to a master server at 127.30.153.190:45223
I20260810 18:53:28.903126 3243 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:28.903769 3243 heartbeater.cc:511] Master 127.30.153.190:45223 requested a full tablet report, sending...
I20260810 18:53:28.905014 2762 ts_manager.cc:194] Registered new tserver with Master: 87231c61fe624c4ca98f3e4f3b95f982 (127.30.153.130:37045)
I20260810 18:53:28.905630 2762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:45129
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:28.930039 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:45223
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:29.049365 3251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:29.049621 3251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:29.049664 3251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:29.053573 3251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:29.053634 3251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:29.053690 3251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:29.058763 3251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:45223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:29.059681 3251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:29.060627 3251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:29.070094 3265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:29.070087 3263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:29.070115 3261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:29.070391 3251 server_base.cc:1034] running on GCE node
I20260810 18:53:29.071000 3251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:29.071621 3251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:29.072816 3251 hybrid_clock.cc:648] HybridClock initialized: now 1786388009072801 us; error 36 us; skew 500 ppm
Aug 10 18:53:29 dist-test-slave-68xk krb5kdc[2695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:29.075737 3251 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:29.076613 3251 webserver.cc:460] Webserver started at http://127.30.153.131:44331/ using document root <none> and password file <none>
I20260810 18:53:29.077178 3251 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:29.077277 3251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:29.077520 3251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:29.079602 3251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "afd08b635e624883bf6cdc90bde013ac"
format_stamp: "Formatted at 2026-08-10 18:53:29 on dist-test-slave-68xk"
server_key: "1d34bb9c33646ce9ce998d10d0c1d4dd"
server_key_iv: "ae2003c24c15c15029f4672ed6125b23"
server_key_version: "encryptionkey@0"
I20260810 18:53:29.080184 3251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "afd08b635e624883bf6cdc90bde013ac"
format_stamp: "Formatted at 2026-08-10 18:53:29 on dist-test-slave-68xk"
server_key: "1d34bb9c33646ce9ce998d10d0c1d4dd"
server_key_iv: "ae2003c24c15c15029f4672ed6125b23"
server_key_version: "encryptionkey@0"
I20260810 18:53:29.083949 3251 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:53:29.086472 3272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:29.087810 3251 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:29.087977 3251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "afd08b635e624883bf6cdc90bde013ac"
format_stamp: "Formatted at 2026-08-10 18:53:29 on dist-test-slave-68xk"
server_key: "1d34bb9c33646ce9ce998d10d0c1d4dd"
server_key_iv: "ae2003c24c15c15029f4672ed6125b23"
server_key_version: "encryptionkey@0"
I20260810 18:53:29.088094 3251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:29.098001 3251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:29.101257 3251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:29.101460 3251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:29.102041 3251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:29.102923 3251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:29.103004 3251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:29.103076 3251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:29.103114 3251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:29.113060 3251 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:35045
I20260810 18:53:29.113087 3431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:35045 every 8 connection(s)
I20260810 18:53:29.114003 3251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:29.117599 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 3251
I20260810 18:53:29.117707 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:29.117996 31334 external_mini_cluster.cc:1428] Setting key 371e91b6194e46c3e4b3a73afaebfef7
Aug 10 18:53:29 dist-test-slave-68xk krb5kdc[2695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:29.128418 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.115992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:50377 (local address 127.30.153.190:45223)
0810 18:53:29.116172 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:29.116177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:29.117087 (+ 910us) server_negotiation.cc:407] Connection header received
0810 18:53:29.118638 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:29.118641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:29.118690 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:29.118793 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:29.120178 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.120761 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.122041 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.122270 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.122439 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.124955 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:29.124965 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:29.124967 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:29.124990 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:29.126369 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:29.126997 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:29.127000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:29.127001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:29.127078 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:29.127461 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:29.127463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:29.127464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:29.127641 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:29.127718 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:29.128194 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 18:53:29.128296 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:53:29.129284 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:29.116404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:45223 (local address 127.30.153.131:50377)
0810 18:53:29.116923 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:29.116966 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:29.118369 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:29.118978 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:29.118986 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:29.119378 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:29.119988 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:29.120003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.120945 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.120951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:29.121907 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:29.121918 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:29.122460 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:29.122470 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:29.122612 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:29.123294 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:29.123311 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:29.124798 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:29.126547 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:29.126552 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:29.126564 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:29.126899 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:29.127222 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:29.127224 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:29.127226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:29.127369 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:29.127745 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:29.127750 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:29.127976 (+ 226us) client_negotiation.cc:769] Sending connection context
0810 18:53:29.128136 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:53:29.128397 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":119,"threads_started":1}
I20260810 18:53:29.130646 3433 heartbeater.cc:347] Connected to a master server at 127.30.153.190:45223
I20260810 18:53:29.130944 3433 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:29.131454 3433 heartbeater.cc:511] Master 127.30.153.190:45223 requested a full tablet report, sending...
I20260810 18:53:29.132565 2762 ts_manager.cc:194] Registered new tserver with Master: afd08b635e624883bf6cdc90bde013ac (127.30.153.131:35045)
I20260810 18:53:29.133132 2762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:50377
I20260810 18:53:29.143882 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:53:29.159883 2822 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:29.158267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58168 (local address 127.30.153.190:45223)
0810 18:53:29.158483 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:29.158487 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:29.158650 (+ 163us) server_negotiation.cc:407] Connection header received
0810 18:53:29.158817 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:29.158820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:29.158875 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:29.158962 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:29.159771 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58168: BlockingRecv error: recv got EOF from 127.0.0.1:58168 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 18:53:29.161150 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 2855
I20260810 18:53:29.185459 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 3052
I20260810 18:53:29.198729 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 3251
I20260810 18:53:29.207120 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 2715
2026-08-10T18:53:29Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2426 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:29 dist-test-slave-68xk krb5kdc[3449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:29 dist-test-slave-68xk krb5kdc[3449](info): set up 2 sockets
Aug 10 18:53:29 dist-test-slave-68xk krb5kdc[3449](info): commencing operation
krb5kdc: starting...
W20260810 18:53:30.526885 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:30 dist-test-slave-68xk krb5kdc[3449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:30.662037 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:34823
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:46305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:34823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:30.782465 3463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:30.782729 3463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:30.782773 3463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:30.786854 3463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:30.786907 3463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:30.786926 3463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:30.786943 3463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:30.786990 3463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:30.792330 3463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:34823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:34823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:30.793363 3463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:30.794466 3463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:30.803869 3474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:30.803893 3476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:30.803988 3471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:30.804656 3463 server_base.cc:1034] running on GCE node
I20260810 18:53:30.805418 3463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:30.806733 3463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:30.807960 3463 hybrid_clock.cc:648] HybridClock initialized: now 1786388010807928 us; error 43 us; skew 500 ppm
Aug 10 18:53:30 dist-test-slave-68xk krb5kdc[3449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:30.814924 3463 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:30.815965 3463 webserver.cc:460] Webserver started at http://127.30.153.190:40835/ using document root <none> and password file <none>
I20260810 18:53:30.816612 3463 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:30.816718 3463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:30.816972 3463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:30.819228 3463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "eff0989aa64a43d2a409c3eec60efc08"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-68xk"
server_key: "a2868a2531d9c22ea0c1d5d35be925f4"
server_key_iv: "5a45724ad9ef50ec6e28416a7f36eb0f"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.819814 3463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "eff0989aa64a43d2a409c3eec60efc08"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-68xk"
server_key: "a2868a2531d9c22ea0c1d5d35be925f4"
server_key_iv: "5a45724ad9ef50ec6e28416a7f36eb0f"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.824033 3463 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:30.826651 3492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:30.827953 3463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:30.828097 3463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "eff0989aa64a43d2a409c3eec60efc08"
format_stamp: "Formatted at 2026-08-10 18:53:30 on dist-test-slave-68xk"
server_key: "a2868a2531d9c22ea0c1d5d35be925f4"
server_key_iv: "5a45724ad9ef50ec6e28416a7f36eb0f"
server_key_version: "encryptionkey@0"
I20260810 18:53:30.828245 3463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:30.858618 3463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:30.862236 3463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:30.862486 3463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:30.870332 3463 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:34823
I20260810 18:53:30.870764 3565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:34823 every 8 connection(s)
I20260810 18:53:30.871415 3463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:30.874275 3566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:30.880100 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 3463
I20260810 18:53:30.880126 3566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08: Bootstrap starting.
I20260810 18:53:30.880254 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:30.880554 31334 external_mini_cluster.cc:1428] Setting key 88aca00f1bf3e8048aebfff971c30fde
I20260810 18:53:30.882997 3566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:30.883803 3566 log.cc:824] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:30.885952 3566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08: No bootstrap required, opened a new log
Aug 10 18:53:30 dist-test-slave-68xk krb5kdc[3449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:30.890568 3566 raft_consensus.cc:348] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff0989aa64a43d2a409c3eec60efc08" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34823 } }
I20260810 18:53:30.890836 3566 raft_consensus.cc:374] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:30.890918 3566 raft_consensus.cc:729] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff0989aa64a43d2a409c3eec60efc08, State: Initialized, Role: FOLLOWER
I20260810 18:53:30.891409 3566 consensus_queue.cc:260] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff0989aa64a43d2a409c3eec60efc08" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34823 } }
I20260810 18:53:30.891567 3566 raft_consensus.cc:388] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:30.891659 3566 raft_consensus.cc:482] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:30.891774 3566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:30.893019 3566 raft_consensus.cc:504] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff0989aa64a43d2a409c3eec60efc08" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34823 } }
I20260810 18:53:30.893374 3566 leader_election.cc:302] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eff0989aa64a43d2a409c3eec60efc08; no voters:
I20260810 18:53:30.893726 3566 leader_election.cc:288] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:30.893846 3574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:30.894081 3574 raft_consensus.cc:686] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [term 1 LEADER]: Becoming Leader. State: Replica: eff0989aa64a43d2a409c3eec60efc08, State: Running, Role: LEADER
I20260810 18:53:30.894464 3574 consensus_queue.cc:237] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff0989aa64a43d2a409c3eec60efc08" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34823 } }
I20260810 18:53:30.895193 3566 sys_catalog.cc:564] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:30.896355 3575 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eff0989aa64a43d2a409c3eec60efc08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff0989aa64a43d2a409c3eec60efc08" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34823 } } }
I20260810 18:53:30.896492 3575 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:30.896569 3571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:30.881930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46510 (local address 127.30.153.190:34823)
0810 18:53:30.882376 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:30.882393 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:30.882454 (+ 61us) server_negotiation.cc:407] Connection header received
0810 18:53:30.883227 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:30.883245 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:30.883587 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:30.883884 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:30.884806 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.885808 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.887035 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.887521 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.887650 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:30.890758 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:30.890763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:30.890776 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:30.890804 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:30.893059 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:30.893578 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:30.893585 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:30.893591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:30.893749 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:30.894107 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:30.894110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:30.894112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:30.894654 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:30.894865 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:30.895179 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 18:53:30.895511 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":152,"threads_started":1}
I20260810 18:53:30.896842 3576 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eff0989aa64a43d2a409c3eec60efc08. Latest consensus state: current_term: 1 leader_uuid: "eff0989aa64a43d2a409c3eec60efc08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eff0989aa64a43d2a409c3eec60efc08" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34823 } } }
I20260810 18:53:30.896920 3576 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:30.899693 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 18:53:30.900779 3598 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:30.900871 3598 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:30.900988 3591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:30.901698 3591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:30.908159 3591 catalog_manager.cc:1347] Generated new cluster ID: 016b6723d9fc42709dce3c98115b9e0a
I20260810 18:53:30.908293 3591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:30.920521 3571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:30.914918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46520 (local address 127.30.153.190:34823)
0810 18:53:30.915061 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:30.915067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:30.915226 (+ 159us) server_negotiation.cc:407] Connection header received
0810 18:53:30.915357 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:30.915360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:30.915407 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:30.915487 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:30.916089 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.916846 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.918036 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:30.918377 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:30.918522 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:30.919481 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:30.919484 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:30.919487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:30.920049 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:30.920083 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:30.920102 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:30.920186 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:30.920318 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 18:53:30.920380 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:30.937860 3591 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:30.939026 3591 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:30.953914 3591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eff0989aa64a43d2a409c3eec60efc08: Generated new TSK 0
I20260810 18:53:30.954767 3591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:30.967027 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 3463
2026-08-10T18:53:30Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1761 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:31 dist-test-slave-68xk krb5kdc[3617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:31 dist-test-slave-68xk krb5kdc[3617](info): set up 2 sockets
Aug 10 18:53:31 dist-test-slave-68xk krb5kdc[3617](info): commencing operation
krb5kdc: starting...
W20260810 18:53:32.248729 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:32 dist-test-slave-68xk krb5kdc[3617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:32.382128 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38813
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:41289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:32.498696 3633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:32.499109 3633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:32.499226 3633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:32.503378 3633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:32.503482 3633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:32.503521 3633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:32.503561 3633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:32.503597 3633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:32.509181 3633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:41289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:32.510304 3633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:32.511397 3633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:32.520681 3645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:32.520714 3639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:32.521025 3643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:32.521456 3633 server_base.cc:1034] running on GCE node
I20260810 18:53:32.522056 3633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:32.523273 3633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:32.524451 3633 hybrid_clock.cc:648] HybridClock initialized: now 1786388012524434 us; error 31 us; skew 500 ppm
Aug 10 18:53:32 dist-test-slave-68xk krb5kdc[3617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388012, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:32.527643 3633 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:32.528601 3633 webserver.cc:460] Webserver started at http://127.30.153.190:44235/ using document root <none> and password file <none>
I20260810 18:53:32.529217 3633 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:32.529287 3633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:32.529515 3633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:32.531708 3633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "25b2fa9cd5bd4005a0ff9e1678a14507"
format_stamp: "Formatted at 2026-08-10 18:53:32 on dist-test-slave-68xk"
server_key: "b07af9242999535e817d7b5f79ef439e"
server_key_iv: "a5b7d9f993b247662699435389648f09"
server_key_version: "encryptionkey@0"
I20260810 18:53:32.532311 3633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "25b2fa9cd5bd4005a0ff9e1678a14507"
format_stamp: "Formatted at 2026-08-10 18:53:32 on dist-test-slave-68xk"
server_key: "b07af9242999535e817d7b5f79ef439e"
server_key_iv: "a5b7d9f993b247662699435389648f09"
server_key_version: "encryptionkey@0"
I20260810 18:53:32.536322 3633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 18:53:32.539206 3656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:32.540696 3633 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:53:32.540844 3633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "25b2fa9cd5bd4005a0ff9e1678a14507"
format_stamp: "Formatted at 2026-08-10 18:53:32 on dist-test-slave-68xk"
server_key: "b07af9242999535e817d7b5f79ef439e"
server_key_iv: "a5b7d9f993b247662699435389648f09"
server_key_version: "encryptionkey@0"
I20260810 18:53:32.540969 3633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:32.561651 3633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:32.565562 3633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:32.565804 3633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:32.574010 3633 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:38813
I20260810 18:53:32.574366 3741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:38813 every 8 connection(s)
I20260810 18:53:32.575232 3633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:32.578225 3745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:32.580436 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 3633
I20260810 18:53:32.580554 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:32.580895 31334 external_mini_cluster.cc:1428] Setting key 9a50d30e03b37974ab57517553c569b4
I20260810 18:53:32.585070 3745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507: Bootstrap starting.
I20260810 18:53:32.587962 3745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:32.588770 3745 log.cc:824] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:53:32 dist-test-slave-68xk krb5kdc[3617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:32.591173 3745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507: No bootstrap required, opened a new log
I20260810 18:53:32.594604 3745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2fa9cd5bd4005a0ff9e1678a14507" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38813 } }
I20260810 18:53:32.594866 3745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:32.594962 3745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25b2fa9cd5bd4005a0ff9e1678a14507, State: Initialized, Role: FOLLOWER
I20260810 18:53:32.595511 3745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2fa9cd5bd4005a0ff9e1678a14507" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38813 } }
I20260810 18:53:32.595687 3745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:32.595779 3745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:32.595923 3745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:32.597126 3745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2fa9cd5bd4005a0ff9e1678a14507" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38813 } }
I20260810 18:53:32.597500 3745 leader_election.cc:302] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25b2fa9cd5bd4005a0ff9e1678a14507; no voters:
I20260810 18:53:32.597679 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.582550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42096 (local address 127.30.153.190:38813)
0810 18:53:32.583056 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:32.583078 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:32.583124 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:53:32.583939 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:32.583965 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:32.584390 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:32.584774 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:32.585611 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.587134 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.588211 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.588758 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.588961 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:32.591794 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:32.591800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:32.591814 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:32.591844 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:32.594251 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:32.594783 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:32.594789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:32.594793 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:32.594965 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:32.595354 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:32.595357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:32.595359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:32.595885 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:32.596106 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:32.596461 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 18:53:32.596797 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":133,"threads_started":1}
I20260810 18:53:32.597872 3745 leader_election.cc:288] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:32.598003 3751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:32.598238 3751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [term 1 LEADER]: Becoming Leader. State: Replica: 25b2fa9cd5bd4005a0ff9e1678a14507, State: Running, Role: LEADER
I20260810 18:53:32.598651 3751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2fa9cd5bd4005a0ff9e1678a14507" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38813 } }
I20260810 18:53:32.599334 3745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:32.600597 3752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25b2fa9cd5bd4005a0ff9e1678a14507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2fa9cd5bd4005a0ff9e1678a14507" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38813 } } }
I20260810 18:53:32.600737 3752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:32.602061 3763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:32.602059 3753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25b2fa9cd5bd4005a0ff9e1678a14507. Latest consensus state: current_term: 1 leader_uuid: "25b2fa9cd5bd4005a0ff9e1678a14507" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b2fa9cd5bd4005a0ff9e1678a14507" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38813 } } }
I20260810 18:53:32.602221 3753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:32.604738 3763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:32.612114 3763 catalog_manager.cc:1347] Generated new cluster ID: 34847980396c405181e881b076b2e717
I20260810 18:53:32.612205 3763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:32.631983 3763 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:32.633515 3763 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:32.647449 3763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25b2fa9cd5bd4005a0ff9e1678a14507: Generated new TSK 0
I20260810 18:53:32.648401 3763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:32.690927 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38813
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:41289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:32.813598 3781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:32.813877 3781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:32.813922 3781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:32.818065 3781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:32.818135 3781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:32.818231 3781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:32.823580 3781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:41289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:32.824698 3781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:32.825801 3781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:32.835779 3791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:32.835779 3788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:32.835790 3787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:32.836148 3781 server_base.cc:1034] running on GCE node
I20260810 18:53:32.836633 3781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:32.837332 3781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:32.838495 3781 hybrid_clock.cc:648] HybridClock initialized: now 1786388012838474 us; error 32 us; skew 500 ppm
Aug 10 18:53:32 dist-test-slave-68xk krb5kdc[3617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388012, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:32.841506 3781 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:32.842555 3781 webserver.cc:460] Webserver started at http://127.30.153.129:42867/ using document root <none> and password file <none>
I20260810 18:53:32.843184 3781 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:32.843255 3781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:32.843530 3781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:32.846206 3781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "14467232dc764e11acdd6b65c97f8e20"
format_stamp: "Formatted at 2026-08-10 18:53:32 on dist-test-slave-68xk"
server_key: "59e248e7372a696497b5628d4a197754"
server_key_iv: "a72f02a8170f034bddd5877c29031a9f"
server_key_version: "encryptionkey@0"
I20260810 18:53:32.847157 3781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "14467232dc764e11acdd6b65c97f8e20"
format_stamp: "Formatted at 2026-08-10 18:53:32 on dist-test-slave-68xk"
server_key: "59e248e7372a696497b5628d4a197754"
server_key_iv: "a72f02a8170f034bddd5877c29031a9f"
server_key_version: "encryptionkey@0"
I20260810 18:53:32.851686 3781 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:53:32.854497 3802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:32.855877 3781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:53:32.856030 3781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "14467232dc764e11acdd6b65c97f8e20"
format_stamp: "Formatted at 2026-08-10 18:53:32 on dist-test-slave-68xk"
server_key: "59e248e7372a696497b5628d4a197754"
server_key_iv: "a72f02a8170f034bddd5877c29031a9f"
server_key_version: "encryptionkey@0"
I20260810 18:53:32.856163 3781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:32.871109 3781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:32.874253 3781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:32.874554 3781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:32.875183 3781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:32.876062 3781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:32.876137 3781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:32.876210 3781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:32.876263 3781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:32.885628 3781 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:45205
I20260810 18:53:32.885661 3975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:45205 every 8 connection(s)
I20260810 18:53:32.886693 3781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:32.889297 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 3781
I20260810 18:53:32.889443 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:32.889814 31334 external_mini_cluster.cc:1428] Setting key 73c862cd1d00434ebd9f48a760335d7e
Aug 10 18:53:32 dist-test-slave-68xk krb5kdc[3617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388012, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:53:32.901760 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.888603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:41879 (local address 127.30.153.190:38813)
0810 18:53:32.888793 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:32.888798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:32.889706 (+ 908us) server_negotiation.cc:407] Connection header received
0810 18:53:32.891376 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:32.891379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:32.891455 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:32.891547 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:32.892758 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.893366 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.894625 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.894897 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.894985 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:32.897799 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:32.897802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:32.897805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:32.897828 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:32.899476 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:32.900086 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:32.900091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:32.900093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:32.900197 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:32.900562 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:32.900567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:32.900569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:32.900761 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:32.900874 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:32.901454 (+ 580us) server_negotiation.cc:299] Negotiation successful
0810 18:53:32.901590 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:53:32.902688 3979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:32.889032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38813 (local address 127.30.153.129:41879)
0810 18:53:32.889526 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:32.889582 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:32.891143 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:32.891743 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:32.891750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:32.892121 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:32.892607 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:32.892620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.893535 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.893538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:32.894497 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:32.894508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:32.895037 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:32.895040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:32.895187 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:32.895815 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:32.895831 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:32.897613 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:32.899604 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:32.899610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:32.899626 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:32.899938 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:32.900304 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:32.900307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:32.900309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:32.900453 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:32.900874 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:32.900883 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:32.901128 (+ 245us) client_negotiation.cc:769] Sending connection context
0810 18:53:32.901410 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 18:53:32.901760 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":151,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
I20260810 18:53:32.904050 3977 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38813
I20260810 18:53:32.904358 3977 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:32.904937 3977 heartbeater.cc:511] Master 127.30.153.190:38813 requested a full tablet report, sending...
I20260810 18:53:32.906715 3682 ts_manager.cc:194] Registered new tserver with Master: 14467232dc764e11acdd6b65c97f8e20 (127.30.153.129:45205)
I20260810 18:53:32.907934 3682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:41879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:32.921380 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38813
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:41289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:33.041054 3983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:33.041325 3983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:33.041415 3983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:33.045497 3983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:33.045564 3983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:33.045624 3983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:33.051276 3983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:41289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:33.052368 3983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:33.053407 3983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:33.063665 3994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:33.063657 3996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:33.063657 3993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:33.064431 3983 server_base.cc:1034] running on GCE node
I20260810 18:53:33.064885 3983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:33.065608 3983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:33.066776 3983 hybrid_clock.cc:648] HybridClock initialized: now 1786388013066757 us; error 29 us; skew 500 ppm
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[3617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:33.069867 3983 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:33.070863 3983 webserver.cc:460] Webserver started at http://127.30.153.130:39757/ using document root <none> and password file <none>
I20260810 18:53:33.071480 3983 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:33.071580 3983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:33.071830 3983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:33.073952 3983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "28e1f1025fcd49e7917f7f06ab5900f3"
format_stamp: "Formatted at 2026-08-10 18:53:33 on dist-test-slave-68xk"
server_key: "febd55c17b9d07f7234e5b428a7a1361"
server_key_iv: "6bb4b2362ce787b8bb99e2d9faf2b49d"
server_key_version: "encryptionkey@0"
I20260810 18:53:33.074627 3983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "28e1f1025fcd49e7917f7f06ab5900f3"
format_stamp: "Formatted at 2026-08-10 18:53:33 on dist-test-slave-68xk"
server_key: "febd55c17b9d07f7234e5b428a7a1361"
server_key_iv: "6bb4b2362ce787b8bb99e2d9faf2b49d"
server_key_version: "encryptionkey@0"
I20260810 18:53:33.078485 3983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:33.080979 4005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:33.082352 3983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:33.082525 3983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "28e1f1025fcd49e7917f7f06ab5900f3"
format_stamp: "Formatted at 2026-08-10 18:53:33 on dist-test-slave-68xk"
server_key: "febd55c17b9d07f7234e5b428a7a1361"
server_key_iv: "6bb4b2362ce787b8bb99e2d9faf2b49d"
server_key_version: "encryptionkey@0"
I20260810 18:53:33.082649 3983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:33.107672 3983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:33.111094 3983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:33.111330 3983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:33.111974 3983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:33.112885 3983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:33.112960 3983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:33.113035 3983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:33.113087 3983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:33.123071 3983 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:38819
I20260810 18:53:33.123100 4174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:38819 every 8 connection(s)
I20260810 18:53:33.124045 3983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:33.129315 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 3983
I20260810 18:53:33.129482 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:33.129825 31334 external_mini_cluster.cc:1428] Setting key d4977feb51b72ddd09647168a050394b
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[3617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:53:33.142261 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:33.126050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:51947 (local address 127.30.153.190:38813)
0810 18:53:33.126228 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:33.126232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:33.128248 (+ 2016us) server_negotiation.cc:407] Connection header received
0810 18:53:33.130098 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:33.130100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:33.130150 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:33.130219 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:33.131880 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.132415 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.134287 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.134599 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.134724 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:33.137846 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:33.137849 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:33.137851 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:33.137877 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:33.139635 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:33.140368 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:33.140372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:33.140373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:33.140464 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:33.140938 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:33.140940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:33.140941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:33.141161 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:33.141246 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:33.142001 (+ 755us) server_negotiation.cc:299] Negotiation successful
0810 18:53:33.142126 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 18:53:33.143415 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:33.126504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38813 (local address 127.30.153.130:51947)
0810 18:53:33.128087 (+ 1583us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:33.128133 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:33.129854 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:33.130398 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:33.130406 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:33.131002 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:33.131714 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:33.131737 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.132562 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.132567 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:33.134130 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:33.134141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.134770 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.134775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:33.134971 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:33.135658 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:33.135676 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:33.137623 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:33.139776 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:33.139787 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:33.139808 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:33.140230 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:33.140578 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:33.140584 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:33.140587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:33.140812 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:33.141294 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:33.141301 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:33.141667 (+ 366us) client_negotiation.cc:769] Sending connection context
0810 18:53:33.141936 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 18:53:33.142324 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1370,"thread_start_us":159,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
I20260810 18:53:33.144773 4176 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38813
I20260810 18:53:33.145084 4176 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:33.145794 4176 heartbeater.cc:511] Master 127.30.153.190:38813 requested a full tablet report, sending...
I20260810 18:53:33.147145 3682 ts_manager.cc:194] Registered new tserver with Master: 28e1f1025fcd49e7917f7f06ab5900f3 (127.30.153.130:38819)
I20260810 18:53:33.147898 3682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:51947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:33.165217 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38813
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:41289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:33.283411 4182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:33.283725 4182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:33.283810 4182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:33.288053 4182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:33.288146 4182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:33.288240 4182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:33.293715 4182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:41289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:33.294879 4182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:33.296209 4182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:33.306881 4190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:33.307052 4192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:33.306953 4189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:33.307185 4182 server_base.cc:1034] running on GCE node
I20260810 18:53:33.307677 4182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:33.308422 4182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:33.309605 4182 hybrid_clock.cc:648] HybridClock initialized: now 1786388013309584 us; error 34 us; skew 500 ppm
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[3617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:33.312697 4182 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:33.313592 4182 webserver.cc:460] Webserver started at http://127.30.153.131:40119/ using document root <none> and password file <none>
I20260810 18:53:33.314154 4182 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:33.314215 4182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:33.314397 4182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:33.316604 4182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "036d3383b85a43c184d06b2e7a0e1cde"
format_stamp: "Formatted at 2026-08-10 18:53:33 on dist-test-slave-68xk"
server_key: "d61e582f85da3d9fcd4fad932dff57f1"
server_key_iv: "dfefaac956eec77bc6b3fd424df538a5"
server_key_version: "encryptionkey@0"
I20260810 18:53:33.317163 4182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "036d3383b85a43c184d06b2e7a0e1cde"
format_stamp: "Formatted at 2026-08-10 18:53:33 on dist-test-slave-68xk"
server_key: "d61e582f85da3d9fcd4fad932dff57f1"
server_key_iv: "dfefaac956eec77bc6b3fd424df538a5"
server_key_version: "encryptionkey@0"
I20260810 18:53:33.321326 4182 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260810 18:53:33.324276 4203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:33.325816 4182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:33.325989 4182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "036d3383b85a43c184d06b2e7a0e1cde"
format_stamp: "Formatted at 2026-08-10 18:53:33 on dist-test-slave-68xk"
server_key: "d61e582f85da3d9fcd4fad932dff57f1"
server_key_iv: "dfefaac956eec77bc6b3fd424df538a5"
server_key_version: "encryptionkey@0"
I20260810 18:53:33.326126 4182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:33.358583 4182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:33.362704 4182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:33.362939 4182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:33.363680 4182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:33.364652 4182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:33.364704 4182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:33.364784 4182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:33.364830 4182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:33.375476 4182 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:43131
I20260810 18:53:33.375501 4377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:43131 every 8 connection(s)
I20260810 18:53:33.376572 4182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:33.383458 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 4182
I20260810 18:53:33.383625 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:33.383970 31334 external_mini_cluster.cc:1428] Setting key fc347205aff017b5e76587b907d57ddb
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[3617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388013, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:33.391990 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:33.378707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:53497 (local address 127.30.153.190:38813)
0810 18:53:33.378867 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:33.378871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:33.379871 (+ 1000us) server_negotiation.cc:407] Connection header received
0810 18:53:33.381269 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:33.381272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:33.381324 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:33.381413 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:33.382790 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.383442 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.384924 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.385395 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.385524 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:33.388414 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:33.388417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:33.388420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:33.388446 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:33.389991 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:33.390574 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:33.390576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:33.390578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:33.390660 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:33.390983 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:33.390985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:33.390987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:33.391165 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:33.391281 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:33.391760 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 18:53:33.391865 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:53:33.392853 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:33.379171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38813 (local address 127.30.153.131:53497)
0810 18:53:33.379678 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:33.379742 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:33.381026 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:33.381586 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:33.381595 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:33.382008 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:33.382589 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:33.382605 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.383598 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.383602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:33.384787 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:33.384798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.385695 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.385698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:33.385867 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:33.386617 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:33.386634 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:33.388252 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:33.390108 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:33.390113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:33.390126 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:33.390470 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:33.390746 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:33.390748 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:33.390750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:33.390892 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:33.391295 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:33.391300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:33.391537 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:53:33.391694 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 18:53:33.391957 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":135,"threads_started":1}
I20260810 18:53:33.394044 4381 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38813
I20260810 18:53:33.394292 4381 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:33.394905 4381 heartbeater.cc:511] Master 127.30.153.190:38813 requested a full tablet report, sending...
I20260810 18:53:33.395985 3682 ts_manager.cc:194] Registered new tserver with Master: 036d3383b85a43c184d06b2e7a0e1cde (127.30.153.131:43131)
I20260810 18:53:33.396595 3682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:53497
I20260810 18:53:33.398288 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[3617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388013, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[3617](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388013, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:33.427412 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:33.418345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.30.153.190:38813)
0810 18:53:33.418599 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:33.418604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:33.418718 (+ 114us) server_negotiation.cc:407] Connection header received
0810 18:53:33.418904 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:33.418907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:33.418961 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:33.419056 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:33.419886 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.420420 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.421298 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:33.421523 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:33.421643 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:33.424251 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:33.424254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:33.424256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:33.424283 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:33.425548 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:33.426019 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:33.426021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:33.426023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:33.426099 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:33.426386 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:33.426389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:33.426390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:33.426709 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:33.426832 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:33.427048 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 18:53:33.427154 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260810 18:53:33.427897 3682 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42106
I20260810 18:53:33.428963 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 3781
I20260810 18:53:33.444371 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 3983
I20260810 18:53:33.463578 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 4182
I20260810 18:53:33.476100 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 3633
2026-08-10T18:53:33Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2508 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[4397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[4397](info): set up 2 sockets
Aug 10 18:53:33 dist-test-slave-68xk krb5kdc[4397](info): commencing operation
krb5kdc: starting...
W20260810 18:53:34.781220 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:34 dist-test-slave-68xk krb5kdc[4397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:34.922884 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:41795
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:41795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:35.038971 4415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:35.039281 4415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:35.039355 4415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:35.043341 4415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:35.043426 4415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:35.043465 4415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:35.043524 4415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:35.043562 4415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:35.049021 4415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:41795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:41795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:35.050096 4415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:35.051200 4415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:35.059860 4428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:35.059842 4422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:35.059842 4423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:35.060245 4415 server_base.cc:1034] running on GCE node
I20260810 18:53:35.060802 4415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:35.061920 4415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:35.063124 4415 hybrid_clock.cc:648] HybridClock initialized: now 1786388015063102 us; error 56 us; skew 500 ppm
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:35.066002 4415 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:35.066985 4415 webserver.cc:460] Webserver started at http://127.30.153.190:43941/ using document root <none> and password file <none>
I20260810 18:53:35.067557 4415 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:35.067655 4415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:35.067901 4415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:35.069965 4415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "e05d1b270fbb4403af15134450b0a5b7"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "eabf0779e9d6fee62cd6a9968d88ee79"
server_key_iv: "85c1bcae09bdaa0455fae59967fb1e8a"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.070608 4415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "e05d1b270fbb4403af15134450b0a5b7"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "eabf0779e9d6fee62cd6a9968d88ee79"
server_key_iv: "85c1bcae09bdaa0455fae59967fb1e8a"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.074302 4415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:35.076680 4438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.077883 4415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:35.078032 4415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "e05d1b270fbb4403af15134450b0a5b7"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "eabf0779e9d6fee62cd6a9968d88ee79"
server_key_iv: "85c1bcae09bdaa0455fae59967fb1e8a"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.078145 4415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:35.100505 4415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:35.103554 4415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:35.103786 4415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:35.111678 4415 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:41795
I20260810 18:53:35.111709 4523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:41795 every 8 connection(s)
I20260810 18:53:35.112651 4415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:35.115577 4524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:35.120357 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 4415
I20260810 18:53:35.120467 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:35.120769 31334 external_mini_cluster.cc:1428] Setting key c0952d53c3fcd4cc06fc83bca7a2c453
I20260810 18:53:35.121441 4524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7: Bootstrap starting.
I20260810 18:53:35.123960 4524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:35.124729 4524 log.cc:824] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:35.126823 4524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7: No bootstrap required, opened a new log
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:35.130862 4524 raft_consensus.cc:348] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05d1b270fbb4403af15134450b0a5b7" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41795 } }
I20260810 18:53:35.131130 4524 raft_consensus.cc:374] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:35.131219 4524 raft_consensus.cc:729] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e05d1b270fbb4403af15134450b0a5b7, State: Initialized, Role: FOLLOWER
I20260810 18:53:35.131770 4524 consensus_queue.cc:260] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05d1b270fbb4403af15134450b0a5b7" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41795 } }
I20260810 18:53:35.131910 4524 raft_consensus.cc:388] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:35.132005 4524 raft_consensus.cc:482] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:35.132114 4524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:35.132980 4524 raft_consensus.cc:504] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05d1b270fbb4403af15134450b0a5b7" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41795 } }
I20260810 18:53:35.133337 4524 leader_election.cc:302] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e05d1b270fbb4403af15134450b0a5b7; no voters:
I20260810 18:53:35.133728 4524 leader_election.cc:288] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:35.133795 4534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:35.134006 4534 raft_consensus.cc:686] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [term 1 LEADER]: Becoming Leader. State: Replica: e05d1b270fbb4403af15134450b0a5b7, State: Running, Role: LEADER
I20260810 18:53:35.134384 4534 consensus_queue.cc:237] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05d1b270fbb4403af15134450b0a5b7" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41795 } }
I20260810 18:53:35.135099 4524 sys_catalog.cc:564] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:35.136217 4536 sys_catalog.cc:455] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e05d1b270fbb4403af15134450b0a5b7. Latest consensus state: current_term: 1 leader_uuid: "e05d1b270fbb4403af15134450b0a5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05d1b270fbb4403af15134450b0a5b7" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41795 } } }
I20260810 18:53:35.136336 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.122269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55198 (local address 127.30.153.190:41795)
0810 18:53:35.122732 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:35.122747 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:35.122782 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:53:35.123411 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:35.123427 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:35.123749 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:35.124081 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:35.124952 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.125947 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.126888 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.127414 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.127591 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.130376 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:35.130381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:35.130394 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:35.130463 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:35.132530 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.132995 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.133001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.133007 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.133193 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.133497 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.133501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.133502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.133986 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:35.134201 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:35.134956 (+ 755us) server_negotiation.cc:299] Negotiation successful
0810 18:53:35.135236 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":325,"thread_start_us":156,"threads_started":1}
I20260810 18:53:35.136346 4536 sys_catalog.cc:458] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:35.136432 4535 sys_catalog.cc:455] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e05d1b270fbb4403af15134450b0a5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e05d1b270fbb4403af15134450b0a5b7" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41795 } } }
I20260810 18:53:35.136785 4535 sys_catalog.cc:458] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7 [sys.catalog]: This master's current role is: LEADER
W20260810 18:53:35.140064 4557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:35.140182 4557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:35.140301 4549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:35.141242 4549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:35.146921 4549 catalog_manager.cc:1347] Generated new cluster ID: c49de7c645e84c9e90e075a76552224d
I20260810 18:53:35.146991 4549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:35.158622 4549 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:35.159488 4549 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:35.170459 4549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e05d1b270fbb4403af15134450b0a5b7: Generated new TSK 0
I20260810 18:53:35.171154 4549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:35.213390 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41795
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:35.329066 4567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:35.329355 4567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:35.329401 4567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:35.333534 4567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:35.333603 4567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:35.333690 4567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:35.339401 4567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:35.340528 4567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:35.341612 4567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:35.351696 4580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:35.351708 4578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:35.351917 4567 server_base.cc:1034] running on GCE node
W20260810 18:53:35.351708 4582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:35.352468 4567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:35.353104 4567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:35.354305 4567 hybrid_clock.cc:648] HybridClock initialized: now 1786388015354289 us; error 27 us; skew 500 ppm
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:35.357518 4567 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:35.358459 4567 webserver.cc:460] Webserver started at http://127.30.153.129:35019/ using document root <none> and password file <none>
I20260810 18:53:35.359025 4567 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:35.359087 4567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:35.359270 4567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:35.361311 4567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "29bf4fe4368049b9a2eeaa9205a83a4c"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "c83a13c07cd0eb4290d5f2cde6d72418"
server_key_iv: "5cd96f961274947a3e9592776e61b204"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.361838 4567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "29bf4fe4368049b9a2eeaa9205a83a4c"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "c83a13c07cd0eb4290d5f2cde6d72418"
server_key_iv: "5cd96f961274947a3e9592776e61b204"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.365734 4567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 18:53:35.368257 4589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.369485 4567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:35.369632 4567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "29bf4fe4368049b9a2eeaa9205a83a4c"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "c83a13c07cd0eb4290d5f2cde6d72418"
server_key_iv: "5cd96f961274947a3e9592776e61b204"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.369727 4567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:35.384578 4567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:35.388149 4567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:35.388391 4567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:35.389051 4567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:35.390101 4567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:35.390164 4567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.390223 4567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:35.390254 4567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.399974 4567 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:36345
I20260810 18:53:35.399994 4762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:36345 every 8 connection(s)
I20260810 18:53:35.400957 4567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:35.411243 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 4567
I20260810 18:53:35.411406 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:35.411747 31334 external_mini_cluster.cc:1428] Setting key e21039ea56fac168baffd8e7ccfd0e32
I20260810 18:53:35.416041 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.403016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:36969 (local address 127.30.153.190:41795)
0810 18:53:35.403167 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:35.403171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:35.404138 (+ 967us) server_negotiation.cc:407] Connection header received
0810 18:53:35.405580 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:35.405584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:35.405647 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:35.405735 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:35.407110 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.407676 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.408871 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.409122 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.409238 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.412113 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:35.412116 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:35.412118 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:35.412145 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:35.413810 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.414366 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.414369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.414371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.414494 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.414878 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.414882 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.414883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.415083 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:35.415199 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:35.415733 (+ 534us) server_negotiation.cc:299] Negotiation successful
0810 18:53:35.415868 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:35.416956 4765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.403431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:41795 (local address 127.30.153.129:36969)
0810 18:53:35.403935 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:35.403980 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:35.405343 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:35.405936 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:35.405943 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:35.406344 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:35.406939 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:35.406954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.407827 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.407830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:35.408722 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:35.408731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.409287 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.409291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:35.409465 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.410106 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:35.410123 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:35.411917 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:35.413954 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:35.413959 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:35.413972 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:35.414245 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:35.414602 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:35.414605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:35.414607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:35.414768 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:35.415200 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:35.415208 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:35.415479 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 18:53:35.415689 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 18:53:35.416063 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":119,"threads_started":1}
I20260810 18:53:35.418467 4763 heartbeater.cc:347] Connected to a master server at 127.30.153.190:41795
I20260810 18:53:35.418789 4763 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:35.419420 4763 heartbeater.cc:511] Master 127.30.153.190:41795 requested a full tablet report, sending...
I20260810 18:53:35.421252 4469 ts_manager.cc:194] Registered new tserver with Master: 29bf4fe4368049b9a2eeaa9205a83a4c (127.30.153.129:36345)
I20260810 18:53:35.422914 4469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:36969
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:35.446997 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41795
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:35.565698 4768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:35.566005 4768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:35.566084 4768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:35.570128 4768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:35.570235 4768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:35.570331 4768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:35.576002 4768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:35.577119 4768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:35.578186 4768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:35.588155 4780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:35.588265 4782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:35.588160 4777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:35.588573 4768 server_base.cc:1034] running on GCE node
I20260810 18:53:35.588970 4768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:35.589581 4768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:35.590773 4768 hybrid_clock.cc:648] HybridClock initialized: now 1786388015590761 us; error 27 us; skew 500 ppm
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:35.593897 4768 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:35.595286 4768 webserver.cc:460] Webserver started at http://127.30.153.130:34047/ using document root <none> and password file <none>
I20260810 18:53:35.595870 4768 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:35.595932 4768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:35.596108 4768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:35.598306 4768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "02f3a3423976491f8b5af8d9c7f74f60"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "9fbc08ff04b45357280d57d996d2f5a9"
server_key_iv: "839d5eb1e46a101b116e6e7cf11d406b"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.598934 4768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "02f3a3423976491f8b5af8d9c7f74f60"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "9fbc08ff04b45357280d57d996d2f5a9"
server_key_iv: "839d5eb1e46a101b116e6e7cf11d406b"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.602865 4768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:35.605687 4795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.607275 4768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:35.607441 4768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "02f3a3423976491f8b5af8d9c7f74f60"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "9fbc08ff04b45357280d57d996d2f5a9"
server_key_iv: "839d5eb1e46a101b116e6e7cf11d406b"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.607573 4768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:35.622679 4768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:35.625941 4768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:35.626174 4768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:35.626888 4768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:35.627799 4768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:35.627882 4768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.627961 4768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:35.628012 4768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.638305 4768 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:44183
I20260810 18:53:35.638362 4961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:44183 every 8 connection(s)
I20260810 18:53:35.639341 4768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:35.645216 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 4768
I20260810 18:53:35.645355 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:35.645646 31334 external_mini_cluster.cc:1428] Setting key b59622d52e9e797d02277df3bcf8df83
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:35.655534 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.641276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:48737 (local address 127.30.153.190:41795)
0810 18:53:35.641431 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:35.641436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:35.642395 (+ 959us) server_negotiation.cc:407] Connection header received
0810 18:53:35.643843 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:35.643847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:35.643904 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:35.644014 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:35.645472 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.646156 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.647684 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.647924 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.648021 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.651425 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:35.651428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:35.651430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:35.651456 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:35.652964 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.653654 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.653657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.653658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.653748 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.654187 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.654189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.654190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.654375 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:35.654519 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:35.655279 (+ 760us) server_negotiation.cc:299] Negotiation successful
0810 18:53:35.655399 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:53:35.656777 4969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.641751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:41795 (local address 127.30.153.130:48737)
0810 18:53:35.642235 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:35.642285 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:35.643624 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:35.644185 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:35.644192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:35.644581 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:35.645176 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:35.645199 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.646307 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.646311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:35.647503 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:35.647518 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.648096 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.648103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:35.648353 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.649006 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:35.649032 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:35.651238 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:35.653094 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:35.653101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:35.653117 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:35.653540 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:35.653856 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:35.653862 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:35.653865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:35.654080 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:35.654543 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:35.654565 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:35.654960 (+ 395us) client_negotiation.cc:769] Sending connection context
0810 18:53:35.655191 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 18:53:35.655592 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:53:35.658079 4963 heartbeater.cc:347] Connected to a master server at 127.30.153.190:41795
I20260810 18:53:35.658365 4963 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:35.658915 4963 heartbeater.cc:511] Master 127.30.153.190:41795 requested a full tablet report, sending...
I20260810 18:53:35.659998 4469 ts_manager.cc:194] Registered new tserver with Master: 02f3a3423976491f8b5af8d9c7f74f60 (127.30.153.130:44183)
I20260810 18:53:35.660617 4469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:48737
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:35.678908 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41795
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:35.799038 4974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:35.799306 4974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:35.799350 4974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:35.803418 4974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:35.803488 4974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:35.803547 4974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:35.809088 4974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:35.810186 4974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:35.811307 4974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:35.821359 4981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:35.821386 4984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:35.821404 4982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:35.821898 4974 server_base.cc:1034] running on GCE node
I20260810 18:53:35.822296 4974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:35.822999 4974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:35.824210 4974 hybrid_clock.cc:648] HybridClock initialized: now 1786388015824176 us; error 49 us; skew 500 ppm
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:35.831748 4974 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:35.832723 4974 webserver.cc:460] Webserver started at http://127.30.153.131:42495/ using document root <none> and password file <none>
I20260810 18:53:35.833333 4974 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:35.833432 4974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:35.833685 4974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:35.836014 4974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "a2bf7054cf51406e8497fddb223e3730"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "5cde72d39162e46329700bd1db8bbd73"
server_key_iv: "eaf93ccb047ac2afb6087152e19ef8ed"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.836659 4974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "a2bf7054cf51406e8497fddb223e3730"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "5cde72d39162e46329700bd1db8bbd73"
server_key_iv: "eaf93ccb047ac2afb6087152e19ef8ed"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.840626 4974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:35.843150 4994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.844421 4974 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:35.844580 4974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "a2bf7054cf51406e8497fddb223e3730"
format_stamp: "Formatted at 2026-08-10 18:53:35 on dist-test-slave-68xk"
server_key: "5cde72d39162e46329700bd1db8bbd73"
server_key_iv: "eaf93ccb047ac2afb6087152e19ef8ed"
server_key_version: "encryptionkey@0"
I20260810 18:53:35.844713 4974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:35.860240 4974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:35.863762 4974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:35.863998 4974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:35.864650 4974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:35.865603 4974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:35.865681 4974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.865780 4974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:35.865834 4974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:35.875563 4974 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:43927
I20260810 18:53:35.875593 5151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:43927 every 8 connection(s)
I20260810 18:53:35.876467 4974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:35.876694 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 4974
I20260810 18:53:35.876838 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:35.877177 31334 external_mini_cluster.cc:1428] Setting key 76f458f9bb48ce49035a21fbf1a19759
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:35.890837 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.878379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:56557 (local address 127.30.153.190:41795)
0810 18:53:35.878537 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:35.878542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:35.879832 (+ 1290us) server_negotiation.cc:407] Connection header received
0810 18:53:35.881185 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:35.881188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:35.881282 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:35.881357 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:35.882753 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.883325 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.884449 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.884662 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.884751 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.887290 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:35.887292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:35.887294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:35.887319 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:35.888800 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.889280 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.889282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.889283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.889356 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.889648 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.889650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.889651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.889835 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:35.889910 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:35.890545 (+ 635us) server_negotiation.cc:299] Negotiation successful
0810 18:53:35.890685 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:53:35.891902 5156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.878988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:41795 (local address 127.30.153.131:56557)
0810 18:53:35.879656 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:35.879717 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:35.880986 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:35.881544 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:35.881557 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:35.881980 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:35.882555 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:35.882570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.883450 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.883453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:35.884336 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:35.884345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.884800 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.884803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:35.884948 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.885549 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:35.885565 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:35.887116 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:35.888913 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:35.888918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:35.888929 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:35.889182 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:35.889434 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:35.889436 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:35.889437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:35.889563 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:35.889941 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:35.889947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:35.890195 (+ 248us) client_negotiation.cc:769] Sending connection context
0810 18:53:35.890455 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 18:53:35.890835 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":161,"threads_started":1}
I20260810 18:53:35.893039 5152 heartbeater.cc:347] Connected to a master server at 127.30.153.190:41795
I20260810 18:53:35.893308 5152 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:35.893818 5152 heartbeater.cc:511] Master 127.30.153.190:41795 requested a full tablet report, sending...
I20260810 18:53:35.894949 4469 ts_manager.cc:194] Registered new tserver with Master: a2bf7054cf51406e8497fddb223e3730 (127.30.153.131:43927)
I20260810 18:53:35.895519 4469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:56557
I20260810 18:53:35.903650 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:35.931124 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.921841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.30.153.129:36345)
0810 18:53:35.922160 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:35.922166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:35.922192 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:53:35.922244 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:35.922248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:35.922497 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:35.922641 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:35.923328 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.924109 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.924927 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.925323 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.925407 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.927864 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:35.927873 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:35.927886 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:35.927909 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:35.929399 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.929795 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.929800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.929801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.929881 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.930115 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.930118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.930119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.930347 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:35.930570 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:35.930770 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 18:53:35.930936 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":116,"threads_started":1}
Aug 10 18:53:35 dist-test-slave-68xk krb5kdc[4397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:35.942458 4531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:35.934028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.30.153.190:41795)
0810 18:53:35.934173 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:35.934178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:35.934242 (+ 64us) server_negotiation.cc:407] Connection header received
0810 18:53:35.934391 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:35.934393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:35.934482 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:35.934570 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:35.935253 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.935844 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.936710 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:35.936974 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:35.937081 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:35.939436 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:35.939440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:35.939444 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:35.939473 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:35.940728 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.941162 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.941167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.941168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.941251 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:35.941527 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:35.941532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:35.941533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:35.941758 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:35.941900 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:35.942132 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 18:53:35.942248 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 18:53:35.942890 4469 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55214
I20260810 18:53:35.943594 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 4567
I20260810 18:53:35.966671 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 4768
I20260810 18:53:35.981334 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 4974
I20260810 18:53:36.004155 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 4415
2026-08-10T18:53:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2522 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:36 dist-test-slave-68xk krb5kdc[5170](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:36 dist-test-slave-68xk krb5kdc[5170](info): set up 2 sockets
Aug 10 18:53:36 dist-test-slave-68xk krb5kdc[5170](info): commencing operation
krb5kdc: starting...
W20260810 18:53:37.314577 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:37 dist-test-slave-68xk krb5kdc[5170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:37.446099 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:35945
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:35945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:37.557097 5187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:37.557384 5187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:37.557461 5187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:37.561460 5187 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:37.561545 5187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:37.561581 5187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:37.561621 5187 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:37.561652 5187 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:37.566816 5187 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:35945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:35945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:37.567836 5187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:37.568866 5187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:37.577342 5196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:37.577358 5202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:37.577442 5187 server_base.cc:1034] running on GCE node
W20260810 18:53:37.577358 5198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:37.578241 5187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:37.579375 5187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:37.580587 5187 hybrid_clock.cc:648] HybridClock initialized: now 1786388017580550 us; error 48 us; skew 500 ppm
Aug 10 18:53:37 dist-test-slave-68xk krb5kdc[5170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388017, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:37.583640 5187 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:37.584342 5187 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:37.584405 5187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:37.584568 5187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:37.586691 5187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4"
format_stamp: "Formatted at 2026-08-10 18:53:37 on dist-test-slave-68xk"
server_key: "93406c59646c1a45432d3ddcb867cf08"
server_key_iv: "dd72d092510ded9b8bf31979738478f8"
server_key_version: "encryptionkey@0"
I20260810 18:53:37.587232 5187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4"
format_stamp: "Formatted at 2026-08-10 18:53:37 on dist-test-slave-68xk"
server_key: "93406c59646c1a45432d3ddcb867cf08"
server_key_iv: "dd72d092510ded9b8bf31979738478f8"
server_key_version: "encryptionkey@0"
I20260810 18:53:37.590915 5187 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:53:37.593436 5209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:37.594792 5187 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:37.594897 5187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4"
format_stamp: "Formatted at 2026-08-10 18:53:37 on dist-test-slave-68xk"
server_key: "93406c59646c1a45432d3ddcb867cf08"
server_key_iv: "dd72d092510ded9b8bf31979738478f8"
server_key_version: "encryptionkey@0"
I20260810 18:53:37.594985 5187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:37.618840 5187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:37.621862 5187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:37.622030 5187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:37.629482 5187 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:35945
I20260810 18:53:37.629851 5287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:35945 every 8 connection(s)
I20260810 18:53:37.630198 5187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:37.632519 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 5187
I20260810 18:53:37.632687 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:37.632972 31334 external_mini_cluster.cc:1428] Setting key b96a46734e46306f690717f6924de522
I20260810 18:53:37.633104 5288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:37.640523 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4: Bootstrap starting.
Aug 10 18:53:37 dist-test-slave-68xk krb5kdc[5170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:37.643000 5288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:37.643975 5288 log.cc:824] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:37.646245 5288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4: No bootstrap required, opened a new log
I20260810 18:53:37.647954 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:37.634368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.30.153.190:35945)
0810 18:53:37.634855 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:37.634876 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:37.634921 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:53:37.635675 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:37.635698 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:37.636112 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:37.636483 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:37.637160 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:37.638670 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:37.639562 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:37.640219 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:37.640387 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:37.642838 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:37.642845 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:37.642865 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:37.642903 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:37.644831 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:37.645309 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:37.645315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:37.645321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:37.645469 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:37.645729 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:37.645732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:37.645734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:37.646217 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:37.646455 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:37.646744 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 18:53:37.647029 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":118,"threads_started":1}
I20260810 18:53:37.650611 5288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35945 } }
I20260810 18:53:37.650830 5288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:37.650866 5288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d55b2d669bf4a43b5f6d681b8a25fd4, State: Initialized, Role: FOLLOWER
I20260810 18:53:37.651212 5288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35945 } }
I20260810 18:53:37.651324 5288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:37.651363 5288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:37.651427 5288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:37.652382 5288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35945 } }
I20260810 18:53:37.652680 5288 leader_election.cc:302] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d55b2d669bf4a43b5f6d681b8a25fd4; no voters:
I20260810 18:53:37.652942 5288 leader_election.cc:288] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:37.653096 5297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:37.653348 5297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 5d55b2d669bf4a43b5f6d681b8a25fd4, State: Running, Role: LEADER
I20260810 18:53:37.653710 5297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35945 } }
I20260810 18:53:37.654151 5288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:37.655577 5299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35945 } } }
I20260810 18:53:37.655727 5299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:37.656038 5300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d55b2d669bf4a43b5f6d681b8a25fd4. Latest consensus state: current_term: 1 leader_uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d55b2d669bf4a43b5f6d681b8a25fd4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35945 } } }
I20260810 18:53:37.656124 5308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:37.656229 5300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:37.658968 5308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:37.664393 5308 catalog_manager.cc:1347] Generated new cluster ID: 75f2516ec0c6446c923ef26f969ac07f
I20260810 18:53:37.664474 5308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:37.676512 5308 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:37.677817 5308 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:37.682006 5308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4: Generated new TSK 0
I20260810 18:53:37.682778 5308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:37.718046 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:35945
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:37.834275 5327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:37.834545 5327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:37.834627 5327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:37.838542 5327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:37.838591 5327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:37.838687 5327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:37.843727 5327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:35945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:37.844951 5327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:37.846067 5327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:37.855764 5338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:37.855756 5335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:37.855757 5336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:37.856191 5327 server_base.cc:1034] running on GCE node
I20260810 18:53:37.856619 5327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:37.857228 5327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:37.858405 5327 hybrid_clock.cc:648] HybridClock initialized: now 1786388017858383 us; error 33 us; skew 500 ppm
Aug 10 18:53:37 dist-test-slave-68xk krb5kdc[5170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388017, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:37.861506 5327 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:37.862277 5327 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:37.862345 5327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:37.862607 5327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:37.864663 5327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "f9c738670d7b488eb9011039e602dbfb"
format_stamp: "Formatted at 2026-08-10 18:53:37 on dist-test-slave-68xk"
server_key: "594b53f9acd863f8cf740b1c9253c797"
server_key_iv: "40a97625daf4ce117bba07041f735b86"
server_key_version: "encryptionkey@0"
I20260810 18:53:37.865218 5327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "f9c738670d7b488eb9011039e602dbfb"
format_stamp: "Formatted at 2026-08-10 18:53:37 on dist-test-slave-68xk"
server_key: "594b53f9acd863f8cf740b1c9253c797"
server_key_iv: "40a97625daf4ce117bba07041f735b86"
server_key_version: "encryptionkey@0"
I20260810 18:53:37.869051 5327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:37.871518 5348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:37.872773 5327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:37.872916 5327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "f9c738670d7b488eb9011039e602dbfb"
format_stamp: "Formatted at 2026-08-10 18:53:37 on dist-test-slave-68xk"
server_key: "594b53f9acd863f8cf740b1c9253c797"
server_key_iv: "40a97625daf4ce117bba07041f735b86"
server_key_version: "encryptionkey@0"
I20260810 18:53:37.873032 5327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:37.894096 5327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:37.897284 5327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:37.897502 5327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:37.898181 5327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:37.899061 5327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:37.899139 5327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:37.899212 5327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:37.899245 5327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:37.909075 5327 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:45481
I20260810 18:53:37.909132 5502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:45481 every 8 connection(s)
I20260810 18:53:37.909727 5327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:37.915531 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 5327
I20260810 18:53:37.915647 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:37.915964 31334 external_mini_cluster.cc:1428] Setting key 736179d386f249d2e55e2136b879edbd
Aug 10 18:53:37 dist-test-slave-68xk krb5kdc[5170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388017, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:37.926304 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:37.911734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:43495 (local address 127.30.153.190:35945)
0810 18:53:37.911889 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:37.911893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:37.912725 (+ 832us) server_negotiation.cc:407] Connection header received
0810 18:53:37.914220 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:37.914224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:37.914294 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:37.914413 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:37.915812 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:37.916553 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:37.917950 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:37.918244 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:37.918337 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:37.921909 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:37.921916 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:37.921922 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:37.921958 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:37.923959 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:37.924511 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:37.924518 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:37.924522 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:37.924637 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:37.925050 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:37.925057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:37.925061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:37.925341 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:37.925468 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:37.925965 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 18:53:37.926119 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:53:37.927124 5506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:37.912065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:35945 (local address 127.30.153.129:43495)
0810 18:53:37.912551 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:37.912597 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:37.913979 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:37.914639 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:37.914652 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:37.915029 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:37.915562 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:37.915575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:37.916707 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:37.916711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:37.917795 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:37.917808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:37.918455 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:37.918463 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:37.918735 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:37.919404 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:37.919433 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:37.921710 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:37.924100 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:37.924105 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:37.924116 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:37.924390 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:37.924778 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:37.924780 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:37.924782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:37.924927 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:37.925466 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:37.925471 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:37.925713 (+ 242us) client_negotiation.cc:769] Sending connection context
0810 18:53:37.925873 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:53:37.926133 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:53:37.928617 5504 heartbeater.cc:347] Connected to a master server at 127.30.153.190:35945
I20260810 18:53:37.928884 5504 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:37.929440 5504 heartbeater.cc:511] Master 127.30.153.190:35945 requested a full tablet report, sending...
I20260810 18:53:37.931150 5234 ts_manager.cc:194] Registered new tserver with Master: f9c738670d7b488eb9011039e602dbfb (127.30.153.129:45481)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
I20260810 18:53:37.932473 5234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:43495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:37.949442 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:35945
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:38.064788 5509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:38.065156 5509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:38.065212 5509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:38.069283 5509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:38.069343 5509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:38.069442 5509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:38.074728 5509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:35945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:38.075846 5509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:38.076973 5509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:38.086373 5516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:38.086524 5515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:38.086373 5520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:38.086653 5509 server_base.cc:1034] running on GCE node
I20260810 18:53:38.087172 5509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:38.087823 5509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:38.089015 5509 hybrid_clock.cc:648] HybridClock initialized: now 1786388018088991 us; error 36 us; skew 500 ppm
Aug 10 18:53:38 dist-test-slave-68xk krb5kdc[5170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:38.091910 5509 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:38.092730 5509 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:38.092823 5509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:38.093083 5509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:38.095302 5509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "bbb6dca6e156466c9dee5b60740fda08"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "c1e829a4cd7daf31887e0cd29bd6d6a8"
server_key_iv: "70ec9374999740a095a260931adbe9d0"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.095912 5509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "bbb6dca6e156466c9dee5b60740fda08"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "c1e829a4cd7daf31887e0cd29bd6d6a8"
server_key_iv: "70ec9374999740a095a260931adbe9d0"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.099815 5509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:38.102262 5527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:38.103605 5509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:38.103744 5509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "bbb6dca6e156466c9dee5b60740fda08"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "c1e829a4cd7daf31887e0cd29bd6d6a8"
server_key_iv: "70ec9374999740a095a260931adbe9d0"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.103854 5509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:38.133301 5509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:38.136754 5509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:38.136978 5509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:38.137615 5509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:38.138602 5509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:38.138675 5509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:38.138749 5509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:38.138793 5509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:38.148332 5509 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:36359
I20260810 18:53:38.148358 5682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:36359 every 8 connection(s)
I20260810 18:53:38.148926 5509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:38.156751 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 5509
I20260810 18:53:38.156888 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:38.157197 31334 external_mini_cluster.cc:1428] Setting key ebc2038ee757851ba25426f8b1fcfc82
Aug 10 18:53:38 dist-test-slave-68xk krb5kdc[5170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:38.164485 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.150716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:40019 (local address 127.30.153.190:35945)
0810 18:53:38.150876 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.150882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.151827 (+ 945us) server_negotiation.cc:407] Connection header received
0810 18:53:38.153264 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.153267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.153317 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.153400 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:38.154834 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.155499 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.156926 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.157193 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.157305 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.160314 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:38.160320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:38.160326 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:38.160361 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:38.161965 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:38.162712 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:38.162718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:38.162723 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:38.162831 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:38.163192 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:38.163198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:38.163203 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:38.163468 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:38.163580 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.164076 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.164232 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:53:38.165220 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.151165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:35945 (local address 127.30.153.130:40019)
0810 18:53:38.151663 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.151707 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:38.152907 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:38.153570 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:38.153577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:38.154073 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:38.154667 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.154682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.155625 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.155628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.156772 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.156787 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.157365 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.157371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.157544 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.158215 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:38.158231 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:38.160143 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:38.162090 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:38.162096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:38.162111 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:38.162606 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:38.162934 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:38.162936 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:38.162937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:38.163080 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:38.163576 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:38.163581 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:38.163805 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 18:53:38.163954 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 18:53:38.164293 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":116,"threads_started":1}
I20260810 18:53:38.166575 5684 heartbeater.cc:347] Connected to a master server at 127.30.153.190:35945
I20260810 18:53:38.166814 5684 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:38.167277 5684 heartbeater.cc:511] Master 127.30.153.190:35945 requested a full tablet report, sending...
I20260810 18:53:38.168386 5234 ts_manager.cc:194] Registered new tserver with Master: bbb6dca6e156466c9dee5b60740fda08 (127.30.153.130:36359)
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:53:38.169047 5234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:40019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:38.189657 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:35945
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:39813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:38.303184 5692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:38.303432 5692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:38.303476 5692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:38.307531 5692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:38.307583 5692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:38.307644 5692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:38.312785 5692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:35945
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:38.313786 5692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:38.314898 5692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:38.324383 5700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:38.324443 5703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:38.324405 5698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:38.324987 5692 server_base.cc:1034] running on GCE node
I20260810 18:53:38.325435 5692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:38.326118 5692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:38.327307 5692 hybrid_clock.cc:648] HybridClock initialized: now 1786388018327263 us; error 57 us; skew 500 ppm
Aug 10 18:53:38 dist-test-slave-68xk krb5kdc[5170](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:38.330109 5692 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:38.330914 5692 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:38.330982 5692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:38.331164 5692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:38.333231 5692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "788025df6cef49829e725d90eae61cef"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "22b2139ede8f6018cfbb861c77636222"
server_key_iv: "db6d1ffd9947ed7ba42da2233deda4e2"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.333752 5692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "788025df6cef49829e725d90eae61cef"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "22b2139ede8f6018cfbb861c77636222"
server_key_iv: "db6d1ffd9947ed7ba42da2233deda4e2"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.337558 5692 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:38.340090 5710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:38.341287 5692 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:38.341413 5692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "788025df6cef49829e725d90eae61cef"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "22b2139ede8f6018cfbb861c77636222"
server_key_iv: "db6d1ffd9947ed7ba42da2233deda4e2"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.341522 5692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:38.353621 5692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:38.357524 5692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:38.357755 5692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:38.358481 5692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:38.359370 5692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:38.359419 5692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:38.359484 5692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:38.359520 5692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:38.369886 5692 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:36337
I20260810 18:53:38.369902 5866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:36337 every 8 connection(s)
I20260810 18:53:38.370612 5692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:38.376672 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 5692
I20260810 18:53:38.376834 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:38.377140 31334 external_mini_cluster.cc:1428] Setting key 089839b4f4a54a32e591ac365d494808
Aug 10 18:53:38 dist-test-slave-68xk krb5kdc[5170](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388018, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:38.385555 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.372465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:53909 (local address 127.30.153.190:35945)
0810 18:53:38.372656 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.372661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.373535 (+ 874us) server_negotiation.cc:407] Connection header received
0810 18:53:38.375095 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.375098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.375149 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.375284 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:38.376636 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.377365 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.378953 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.379283 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.379393 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.381929 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:38.381932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:38.381934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:38.381959 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:38.383474 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:38.384077 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:38.384080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:38.384081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:38.384157 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:38.384503 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:38.384505 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:38.384506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:38.384673 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:38.384750 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.385326 (+ 576us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.385425 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:53:38.386526 5869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.372911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:35945 (local address 127.30.153.131:53909)
0810 18:53:38.373379 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.373422 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:38.374825 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:38.375467 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:38.375474 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:38.375881 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:38.376453 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.376476 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.377508 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.377511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.378840 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.378849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.379479 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.379484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.379682 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.380210 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:38.380226 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:38.381774 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:38.383595 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:38.383599 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:38.383618 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:38.383960 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:38.384240 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:38.384243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:38.384244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:38.384417 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:38.384786 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:38.384794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:38.385079 (+ 285us) client_negotiation.cc:769] Sending connection context
0810 18:53:38.385251 (+ 172us) client_negotiation.cc:240] Negotiation successful
0810 18:53:38.385508 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":114,"threads_started":1}
I20260810 18:53:38.387645 5867 heartbeater.cc:347] Connected to a master server at 127.30.153.190:35945
I20260810 18:53:38.387880 5867 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:38.388342 5867 heartbeater.cc:511] Master 127.30.153.190:35945 requested a full tablet report, sending...
I20260810 18:53:38.389328 5234 ts_manager.cc:194] Registered new tserver with Master: 788025df6cef49829e725d90eae61cef (127.30.153.131:36337)
I20260810 18:53:38.390013 5234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:53909
I20260810 18:53:38.391839 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:38.402215 5295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.393844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.30.153.190:35945)
0810 18:53:38.393969 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.393973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.394736 (+ 763us) server_negotiation.cc:407] Connection header received
0810 18:53:38.394862 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.394864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.394902 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.394968 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:38.395968 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.396476 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.397397 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.397626 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.397707 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.399236 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:38.399240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:38.399242 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:38.399269 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:38.400524 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:38.400955 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:38.400960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:38.400961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:38.401043 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:38.401330 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:38.401334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:38.401335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:38.401548 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:38.401694 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.401932 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.402051 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:53:38.405568 5234 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:38.408963 5234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:38.424288 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.417845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:36337 (local address 127.0.0.1:33014)
0810 18:53:38.418752 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.418810 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:38.418941 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:38.419589 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:38.419594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:38.419618 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:38.419761 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.419768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.421095 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.421102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.422577 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.422587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.423723 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.423729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.423836 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.423872 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:53:38.423987 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 18:53:38.424116 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":111,"thread_start_us":117,"threads_started":1}
I20260810 18:53:38.424333 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.418101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.30.153.131:36337)
0810 18:53:38.418557 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.418570 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.419106 (+ 536us) server_negotiation.cc:407] Connection header received
0810 18:53:38.419195 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.419204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.419422 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.419591 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:38.419907 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.420948 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.422707 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.423572 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.423715 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.423847 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.424035 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.424150 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":125,"threads_started":1}
I20260810 18:53:38.424526 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.418121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50726 (local address 127.30.153.129:45481)
0810 18:53:38.418549 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.418556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.418907 (+ 351us) server_negotiation.cc:407] Connection header received
0810 18:53:38.419163 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.419170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.419388 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.419525 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:38.420023 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.421038 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.422853 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.423599 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.423719 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.423848 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.424261 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.424367 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":87,"threads_started":1}
I20260810 18:53:38.424692 5886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.419197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41292 (local address 127.30.153.130:36359)
0810 18:53:38.419937 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.419951 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.419976 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:53:38.420050 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.420054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.420237 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.420389 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:38.420637 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.421724 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.423095 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.423665 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.423790 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.423916 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.424438 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.424542 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":108,"threads_started":1}
I20260810 18:53:38.424933 5881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.417767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45481 (local address 127.0.0.1:50726)
0810 18:53:38.418756 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.418810 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:38.418991 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:38.419757 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:38.419760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:38.419781 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:38.419885 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.419891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.421180 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.421183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.422721 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.422732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.423924 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.423927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.424050 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.424114 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 18:53:38.424688 (+ 574us) client_negotiation.cc:240] Negotiation successful
0810 18:53:38.424767 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":227,"thread_start_us":76,"threads_started":1}
I20260810 18:53:38.424337 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.419535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:36359 (local address 127.0.0.1:41292)
0810 18:53:38.419855 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.419871 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:38.419953 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:38.420393 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:38.420396 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:38.420416 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:38.420493 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.420499 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.421874 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.421878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.422975 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.422982 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.423984 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.423987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.424085 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.424110 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:53:38.424174 (+ 64us) client_negotiation.cc:240] Negotiation successful
0810 18:53:38.424229 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260810 18:53:38.428133 5780 tablet_service.cc:1467] Processing CreateTablet for tablet 126b46633c0d494280d897b5a539b1d1 (DEFAULT_TABLE table=test-table [id=5f24b9a3a7534984afecad0be23178a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:38.428457 5410 tablet_service.cc:1467] Processing CreateTablet for tablet 126b46633c0d494280d897b5a539b1d1 (DEFAULT_TABLE table=test-table [id=5f24b9a3a7534984afecad0be23178a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:38.429221 5780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 126b46633c0d494280d897b5a539b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:38.429517 5410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 126b46633c0d494280d897b5a539b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:38.429175 5593 tablet_service.cc:1467] Processing CreateTablet for tablet 126b46633c0d494280d897b5a539b1d1 (DEFAULT_TABLE table=test-table [id=5f24b9a3a7534984afecad0be23178a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:38.430184 5593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 126b46633c0d494280d897b5a539b1d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:38.436088 5889 tablet_bootstrap.cc:492] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Bootstrap starting.
I20260810 18:53:38.437091 5890 tablet_bootstrap.cc:492] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb: Bootstrap starting.
I20260810 18:53:38.437704 5893 tablet_bootstrap.cc:492] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: Bootstrap starting.
I20260810 18:53:38.438917 5889 tablet_bootstrap.cc:654] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:38.439291 5890 tablet_bootstrap.cc:654] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:38.439877 5893 tablet_bootstrap.cc:654] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:38.440081 5890 log.cc:824] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:38.440310 5889 log.cc:824] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:38.441156 5893 log.cc:824] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:38.442282 5890 tablet_bootstrap.cc:492] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb: No bootstrap required, opened a new log
I20260810 18:53:38.442548 5890 ts_tablet_manager.cc:1397] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:53:38.443027 5889 tablet_bootstrap.cc:492] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: No bootstrap required, opened a new log
I20260810 18:53:38.443025 5893 tablet_bootstrap.cc:492] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: No bootstrap required, opened a new log
I20260810 18:53:38.443209 5889 ts_tablet_manager.cc:1397] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:53:38.443256 5893 ts_tablet_manager.cc:1397] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260810 18:53:38.446324 5890 raft_consensus.cc:348] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.446625 5890 raft_consensus.cc:374] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:38.446684 5890 raft_consensus.cc:729] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c738670d7b488eb9011039e602dbfb, State: Initialized, Role: FOLLOWER
I20260810 18:53:38.447126 5890 consensus_queue.cc:260] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.447124 5893 raft_consensus.cc:348] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.447386 5893 raft_consensus.cc:374] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:38.447441 5893 raft_consensus.cc:729] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb6dca6e156466c9dee5b60740fda08, State: Initialized, Role: FOLLOWER
I20260810 18:53:38.447352 5889 raft_consensus.cc:348] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.447620 5889 raft_consensus.cc:374] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:38.447674 5889 raft_consensus.cc:729] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 788025df6cef49829e725d90eae61cef, State: Initialized, Role: FOLLOWER
I20260810 18:53:38.447911 5893 consensus_queue.cc:260] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.448076 5889 consensus_queue.cc:260] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.448205 5890 ts_tablet_manager.cc:1428] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:53:38.448559 5504 heartbeater.cc:503] Master 127.30.153.190:35945 was elected leader, sending a full tablet report...
I20260810 18:53:38.448876 5889 ts_tablet_manager.cc:1428] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:53:38.448866 5893 ts_tablet_manager.cc:1428] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260810 18:53:38.449204 5867 heartbeater.cc:503] Master 127.30.153.190:35945 was elected leader, sending a full tablet report...
I20260810 18:53:38.449204 5684 heartbeater.cc:503] Master 127.30.153.190:35945 was elected leader, sending a full tablet report...
I20260810 18:53:38.540639 5900 raft_consensus.cc:482] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:38.540853 5900 raft_consensus.cc:504] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.542042 5900 leader_election.cc:288] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9c738670d7b488eb9011039e602dbfb (127.30.153.129:45481), bbb6dca6e156466c9dee5b60740fda08 (127.30.153.130:36359)
I20260810 18:53:38.546586 5869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.542296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:36359 (local address 127.30.153.131:51727)
0810 18:53:38.542473 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.542490 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:38.542583 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:38.543153 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:38.543157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:38.543179 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:38.543343 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.543355 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.544280 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.544284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.545219 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.545229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.546149 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.546153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.546262 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.546278 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 18:53:38.546377 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 18:53:38.546463 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":21}
I20260810 18:53:38.546586 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.542564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:53941 (local address 127.30.153.129:45481)
0810 18:53:38.542713 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.542716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.543130 (+ 414us) server_negotiation.cc:407] Connection header received
0810 18:53:38.543197 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.543200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.543244 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.543368 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:38.543739 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.544272 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.545584 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.545988 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.546109 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.546166 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.546353 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.546452 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:53:38.546625 5886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.542561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:51727 (local address 127.30.153.130:36359)
0810 18:53:38.542767 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.542772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.542813 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:53:38.542883 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.542887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.542954 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.543059 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:38.543492 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.544150 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.545355 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.545986 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.546141 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.546197 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.546444 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.546520 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260810 18:53:38.546613 5901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.542332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45481 (local address 127.30.153.131:53941)
0810 18:53:38.542875 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.542890 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:38.543008 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:38.543458 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:38.543462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:38.543484 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:38.543610 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.543617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.544410 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.544412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.545435 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:38.545446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.546109 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.546113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:38.546205 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.546235 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 18:53:38.546307 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 18:53:38.546399 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":23,"thread_start_us":179,"threads_started":1}
I20260810 18:53:38.547253 5618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126b46633c0d494280d897b5a539b1d1" candidate_uuid: "788025df6cef49829e725d90eae61cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbb6dca6e156466c9dee5b60740fda08" is_pre_election: true
I20260810 18:53:38.547412 5441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126b46633c0d494280d897b5a539b1d1" candidate_uuid: "788025df6cef49829e725d90eae61cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9c738670d7b488eb9011039e602dbfb" is_pre_election: true
I20260810 18:53:38.547603 5618 raft_consensus.cc:2454] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 788025df6cef49829e725d90eae61cef in term 0.
I20260810 18:53:38.547745 5441 raft_consensus.cc:2454] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 788025df6cef49829e725d90eae61cef in term 0.
I20260810 18:53:38.548151 5711 leader_election.cc:302] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788025df6cef49829e725d90eae61cef, bbb6dca6e156466c9dee5b60740fda08; no voters:
I20260810 18:53:38.548420 5900 raft_consensus.cc:2781] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:38.548534 5900 raft_consensus.cc:482] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:38.548605 5900 raft_consensus.cc:3037] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:38.567732 5900 raft_consensus.cc:504] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.568382 5900 leader_election.cc:288] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [CANDIDATE]: Term 1 election: Requested vote from peers f9c738670d7b488eb9011039e602dbfb (127.30.153.129:45481), bbb6dca6e156466c9dee5b60740fda08 (127.30.153.130:36359)
I20260810 18:53:38.568929 5618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126b46633c0d494280d897b5a539b1d1" candidate_uuid: "788025df6cef49829e725d90eae61cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbb6dca6e156466c9dee5b60740fda08"
I20260810 18:53:38.568953 5441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "126b46633c0d494280d897b5a539b1d1" candidate_uuid: "788025df6cef49829e725d90eae61cef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9c738670d7b488eb9011039e602dbfb"
I20260810 18:53:38.569106 5618 raft_consensus.cc:3037] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:38.569139 5441 raft_consensus.cc:3037] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:38.570863 5441 raft_consensus.cc:2454] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788025df6cef49829e725d90eae61cef in term 1.
I20260810 18:53:38.571014 5618 raft_consensus.cc:2454] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 788025df6cef49829e725d90eae61cef in term 1.
I20260810 18:53:38.571341 5712 leader_election.cc:302] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 788025df6cef49829e725d90eae61cef, f9c738670d7b488eb9011039e602dbfb; no voters:
I20260810 18:53:38.571549 5900 raft_consensus.cc:2781] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:38.571898 5900 raft_consensus.cc:686] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 1 LEADER]: Becoming Leader. State: Replica: 788025df6cef49829e725d90eae61cef, State: Running, Role: LEADER
I20260810 18:53:38.572337 5900 consensus_queue.cc:237] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } }
I20260810 18:53:38.576200 5233 catalog_manager.cc:5495] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef reported cstate change: term changed from 0 to 1, leader changed from <none> to 788025df6cef49829e725d90eae61cef (127.30.153.131). New cstate: current_term: 1 leader_uuid: "788025df6cef49829e725d90eae61cef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "788025df6cef49829e725d90eae61cef" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 36337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:38.616505 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.612737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.30.153.131:36337)
0810 18:53:38.612888 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.612894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.612912 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:53:38.613003 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.613005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.613060 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.613190 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:38.613455 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.613963 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.614922 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:38.615163 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:38.615274 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:38.615606 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:38.616043 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:38.616178 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.616272 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.616332 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 18:53:38.621598 5868 tablet.cc:2369] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:38.622701 5441 raft_consensus.cc:1261] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb [term 1 FOLLOWER]: Refusing update from remote peer 788025df6cef49829e725d90eae61cef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:38.622715 5618 raft_consensus.cc:1261] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 1 FOLLOWER]: Refusing update from remote peer 788025df6cef49829e725d90eae61cef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:38.623479 5902 consensus_queue.cc:1035] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbb6dca6e156466c9dee5b60740fda08" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 36359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:38.623658 5900 consensus_queue.cc:1035] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9c738670d7b488eb9011039e602dbfb" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:38.635429 5909 mvcc.cc:205] Tried to move back new op lower bound from 7317045324270706688 to 7317045324075917312. Current Snapshot: MvccSnapshot[applied={T|T < 7317045324270706688}]
I20260810 18:53:38.644928 5232 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:38.645187 5232 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:38.647945 5232 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5d55b2d669bf4a43b5f6d681b8a25fd4: Sending DeleteTablet for 3 replicas of tablet 126b46633c0d494280d897b5a539b1d1
I20260810 18:53:38.648797 5410 tablet_service.cc:1514] Processing DeleteTablet for tablet 126b46633c0d494280d897b5a539b1d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:38 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:50726
I20260810 18:53:38.649416 5922 tablet_replica.cc:330] T 126b46633c0d494280d897b5a539b1d1 P f9c738670d7b488eb9011039e602dbfb: stopping tablet replica
I20260810 18:53:38.649487 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 5327
I20260810 18:53:38.649773 5593 tablet_service.cc:1514] Processing DeleteTablet for tablet 126b46633c0d494280d897b5a539b1d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:38 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:41292
I20260810 18:53:38.650153 5780 tablet_service.cc:1514] Processing DeleteTablet for tablet 126b46633c0d494280d897b5a539b1d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:38 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33014
I20260810 18:53:38.650288 5923 tablet_replica.cc:330] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: stopping tablet replica
I20260810 18:53:38.650619 5925 tablet_replica.cc:330] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: stopping tablet replica
I20260810 18:53:38.650923 5925 raft_consensus.cc:2229] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:38.651450 5925 raft_consensus.cc:2258] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:38.651886 5923 raft_consensus.cc:2229] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:38.652218 5923 raft_consensus.cc:2258] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:38.656594 5923 ts_tablet_manager.cc:1905] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:38.659543 5925 ts_tablet_manager.cc:1905] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:38.672243 5923 ts_tablet_manager.cc:1918] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:38.672503 5923 log.cc:1196] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/126b46633c0d494280d897b5a539b1d1
I20260810 18:53:38.672957 5923 ts_tablet_manager.cc:1939] T 126b46633c0d494280d897b5a539b1d1 P bbb6dca6e156466c9dee5b60740fda08: Deleting consensus metadata
W20260810 18:53:38.673377 5214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:45481 (error 108)
I20260810 18:53:38.674217 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 5509
I20260810 18:53:38.689467 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.689091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45481 (local address 127.0.0.1:50738)
0810 18:53:38.689275 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.689376 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:45481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260810 18:53:38.689762 5214 catalog_manager.cc:4584] TS f9c738670d7b488eb9011039e602dbfb (127.30.153.129:45481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 126b46633c0d494280d897b5a539b1d1: Network error: Client connection negotiation failed: client connection to 127.30.153.129:45481: connect: Connection refused (error 111)
I20260810 18:53:38.697088 5925 ts_tablet_manager.cc:1918] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:38.697296 5925 log.cc:1196] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/126b46633c0d494280d897b5a539b1d1
I20260810 18:53:38.697801 5925 ts_tablet_manager.cc:1939] T 126b46633c0d494280d897b5a539b1d1 P 788025df6cef49829e725d90eae61cef: Deleting consensus metadata
I20260810 18:53:38.704233 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 5692
I20260810 18:53:38.706987 5883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.704451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:36359 (local address 127.0.0.1:41306)
0810 18:53:38.706807 (+ 2356us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:38.706878 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.130:36359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2238}
I20260810 18:53:38.735526 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 5187
2026-08-10T18:53:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2740 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T18:53:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:38Z Disabled control of system clock
I20260810 18:53:38.779876 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38969
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:53:38.904516 5931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:38.904824 5931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:38.904861 5931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:38.909058 5931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:38.909128 5931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:38.909145 5931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:38.909162 5931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:38.909178 5931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:38.914438 5931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:38.915580 5931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:38.917090 5931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:38.926431 5942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:38.926460 5931 server_base.cc:1034] running on GCE node
W20260810 18:53:38.926373 5939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:38.926384 5937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:38.927301 5931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:38.928500 5931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:38.929685 5931 hybrid_clock.cc:648] HybridClock initialized: now 1786388018929648 us; error 52 us; skew 500 ppm
I20260810 18:53:38.932050 5931 webserver.cc:460] Webserver started at http://127.30.153.190:32783/ using document root <none> and password file <none>
I20260810 18:53:38.932708 5931 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:38.932777 5931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:38.933012 5931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:38.935268 5931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "cd2105d9958548debafa45ca1bda596a"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "4a1fafa30ab3b78b47977878984320d6"
server_key_iv: "8bc89b783d0fe58074ce6522f6d7ffd8"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.935880 5931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "cd2105d9958548debafa45ca1bda596a"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "4a1fafa30ab3b78b47977878984320d6"
server_key_iv: "8bc89b783d0fe58074ce6522f6d7ffd8"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.939710 5931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:38.942225 5948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:38.943527 5931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:38.943678 5931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "cd2105d9958548debafa45ca1bda596a"
format_stamp: "Formatted at 2026-08-10 18:53:38 on dist-test-slave-68xk"
server_key: "4a1fafa30ab3b78b47977878984320d6"
server_key_iv: "8bc89b783d0fe58074ce6522f6d7ffd8"
server_key_version: "encryptionkey@0"
I20260810 18:53:38.943805 5931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:38.948916 5931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:38.949529 5931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:38.949716 5931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:38.957336 5931 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:38969
I20260810 18:53:38.957372 6022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:38969 every 8 connection(s)
I20260810 18:53:38.958318 5931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:38.961681 6023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:38.966562 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 5931
I20260810 18:53:38.966706 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:38.967060 31334 external_mini_cluster.cc:1428] Setting key 6035858920999da16dbd5252b2690afc
I20260810 18:53:38.967937 6023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a: Bootstrap starting.
I20260810 18:53:38.970603 6023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:38.971553 6023 log.cc:824] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:38.973549 6023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a: No bootstrap required, opened a new log
I20260810 18:53:38.973656 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:38.968720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.30.153.190:38969)
0810 18:53:38.969488 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:38.969506 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:38.969544 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:53:38.970267 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:38.970286 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:38.970753 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:38.971192 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:38.972002 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:38.972008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:38.972030 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:38.972040 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:38.972095 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:38.972111 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:38.972240 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:38.972344 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 18:53:38.972643 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":141,"threads_started":1}
I20260810 18:53:38.977620 6023 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2105d9958548debafa45ca1bda596a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38969 } }
I20260810 18:53:38.977866 6023 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:38.977928 6023 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2105d9958548debafa45ca1bda596a, State: Initialized, Role: FOLLOWER
I20260810 18:53:38.978345 6023 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2105d9958548debafa45ca1bda596a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38969 } }
I20260810 18:53:38.978543 6023 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:38.978636 6023 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:38.978735 6023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:39.075040 6023 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2105d9958548debafa45ca1bda596a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38969 } }
I20260810 18:53:39.075673 6023 leader_election.cc:302] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd2105d9958548debafa45ca1bda596a; no voters:
I20260810 18:53:39.076153 6023 leader_election.cc:288] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:39.076233 6031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:39.076476 6031 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [term 1 LEADER]: Becoming Leader. State: Replica: cd2105d9958548debafa45ca1bda596a, State: Running, Role: LEADER
I20260810 18:53:39.076850 6031 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2105d9958548debafa45ca1bda596a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38969 } }
I20260810 18:53:39.077728 6023 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:39.078182 6032 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd2105d9958548debafa45ca1bda596a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2105d9958548debafa45ca1bda596a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38969 } } }
I20260810 18:53:39.078344 6032 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:39.078884 6036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:39.080114 6033 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd2105d9958548debafa45ca1bda596a. Latest consensus state: current_term: 1 leader_uuid: "cd2105d9958548debafa45ca1bda596a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2105d9958548debafa45ca1bda596a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38969 } } }
I20260810 18:53:39.080235 6033 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:39.083166 6036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:39.088631 6036 catalog_manager.cc:1347] Generated new cluster ID: 14027e3abf7a4807afa5fde4922ae767
I20260810 18:53:39.088706 6036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:39.111248 6036 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:39.111595 6036 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:39.119803 6036 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a: Generated new TSK 0
I20260810 18:53:39.120640 6036 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:39.128185 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38969
--builtin_ntp_servers=127.30.153.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:53:39.246570 6056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:39.246883 6056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:39.246956 6056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:39.251163 6056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:39.251250 6056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:39.251343 6056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:39.256364 6056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:39.257570 6056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:39.259274 6056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:39.269349 6066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:39.269349 6068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:39.269522 6056 server_base.cc:1034] running on GCE node
W20260810 18:53:39.269538 6064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:39.270099 6056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:39.270874 6056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:39.272101 6056 hybrid_clock.cc:648] HybridClock initialized: now 1786388019272042 us; error 67 us; skew 500 ppm
I20260810 18:53:39.274180 6056 webserver.cc:460] Webserver started at http://127.30.153.129:38143/ using document root <none> and password file <none>
I20260810 18:53:39.274869 6056 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:39.274969 6056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:39.275210 6056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:39.277454 6056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "faef10ffb9cb47ff9c8779acd31cd5a0"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "502908f3a8366b2c228370f5aff62592"
server_key_iv: "6de770184c01ca2dbffd8f0d3ecf94e9"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.278111 6056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "faef10ffb9cb47ff9c8779acd31cd5a0"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "502908f3a8366b2c228370f5aff62592"
server_key_iv: "6de770184c01ca2dbffd8f0d3ecf94e9"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.282316 6056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:53:39.285043 6075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.286541 6056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:39.286691 6056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "faef10ffb9cb47ff9c8779acd31cd5a0"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "502908f3a8366b2c228370f5aff62592"
server_key_iv: "6de770184c01ca2dbffd8f0d3ecf94e9"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.286824 6056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:39.291762 6056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:39.292375 6056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:39.292568 6056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:39.293259 6056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:39.294190 6056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:39.294265 6056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.294334 6056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:39.294385 6056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.303829 6056 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:42391
I20260810 18:53:39.303908 6235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:42391 every 8 connection(s)
I20260810 18:53:39.304837 6056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:39.305955 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 6056
I20260810 18:53:39.306146 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:39.306531 31334 external_mini_cluster.cc:1428] Setting key 7a0322d9821c410608a95adf85dc0fb8
I20260810 18:53:39.309039 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38969
--builtin_ntp_servers=127.30.153.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 18:53:39.312798 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.307161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:54589 (local address 127.30.153.190:38969)
0810 18:53:39.307335 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.307339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.308760 (+ 1421us) server_negotiation.cc:407] Connection header received
0810 18:53:39.310913 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.310916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.310973 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.311073 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.312319 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.312325 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.312329 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.312347 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.312380 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.312392 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.312496 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.312623 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.312690 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:53:39.313687 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.307678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38969 (local address 127.30.153.129:54589)
0810 18:53:39.308583 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.308638 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:39.310673 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:39.311308 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:39.311315 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:39.311405 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:39.312038 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:39.312039 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:39.312081 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:39.312149 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:39.312484 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:39.312491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:39.312495 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:53:39.312634 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 18:53:39.312833 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":152,"threads_started":1}
I20260810 18:53:39.315037 6236 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38969
I20260810 18:53:39.315286 6236 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:39.315716 6236 heartbeater.cc:511] Master 127.30.153.190:38969 requested a full tablet report, sending...
I20260810 18:53:39.317530 5976 ts_manager.cc:194] Registered new tserver with Master: faef10ffb9cb47ff9c8779acd31cd5a0 (127.30.153.129:42391)
W20260810 18:53:39.433439 6241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:39.433702 6241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:39.433743 6241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:39.437727 6241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:39.437788 6241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:39.437844 6241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:39.442584 6241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:39.443671 6241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:39.445055 6241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:39.456085 6246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:39.456130 6247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:39.456125 6249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:39.456228 6241 server_base.cc:1034] running on GCE node
I20260810 18:53:39.456776 6241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:39.457437 6241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:39.458626 6241 hybrid_clock.cc:648] HybridClock initialized: now 1786388019458608 us; error 38 us; skew 500 ppm
I20260810 18:53:39.460492 6241 webserver.cc:460] Webserver started at http://127.30.153.130:34817/ using document root <none> and password file <none>
I20260810 18:53:39.461086 6241 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:39.461189 6241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:39.461424 6241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:39.463526 6241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "6a7bf51f87774a079fcb773a873256b9"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "c58853906029a7a60bebc601f04a0da0"
server_key_iv: "64efb68a04bc3957472b772b1c13cb89"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.464123 6241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "6a7bf51f87774a079fcb773a873256b9"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "c58853906029a7a60bebc601f04a0da0"
server_key_iv: "64efb68a04bc3957472b772b1c13cb89"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.467921 6241 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:39.470706 6259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.472028 6241 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:39.472214 6241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "6a7bf51f87774a079fcb773a873256b9"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "c58853906029a7a60bebc601f04a0da0"
server_key_iv: "64efb68a04bc3957472b772b1c13cb89"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.472344 6241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:39.477151 6241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:39.477821 6241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:39.478011 6241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:39.478670 6241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:39.479699 6241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:39.479782 6241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.479854 6241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:39.479887 6241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.489476 6241 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:37611
I20260810 18:53:39.489521 6419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:37611 every 8 connection(s)
I20260810 18:53:39.490486 6241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:39.496922 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.492406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:38787 (local address 127.30.153.190:38969)
0810 18:53:39.492558 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.492562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.493614 (+ 1052us) server_negotiation.cc:407] Connection header received
0810 18:53:39.495084 (+ 1470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.495087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.495132 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.495232 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.496413 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.496417 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.496420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.496433 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.496456 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.496474 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.496573 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.496730 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.496779 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:39.497803 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.492792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38969 (local address 127.30.153.130:38787)
0810 18:53:39.493463 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.493502 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:39.494865 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:39.495393 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:39.495400 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:39.495488 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:39.496123 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:39.496125 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:39.496167 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:39.496238 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:39.496567 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:39.496578 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:39.496586 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:53:39.496780 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 18:53:39.497024 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":117,"threads_started":1}
I20260810 18:53:39.499008 6420 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38969
I20260810 18:53:39.499209 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 6241
I20260810 18:53:39.499275 6420 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:39.499363 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:39.499662 31334 external_mini_cluster.cc:1428] Setting key efa279ba4a038d8c21c1ec2bda60278a
I20260810 18:53:39.499723 6420 heartbeater.cc:511] Master 127.30.153.190:38969 requested a full tablet report, sending...
I20260810 18:53:39.500936 5972 ts_manager.cc:194] Registered new tserver with Master: 6a7bf51f87774a079fcb773a873256b9 (127.30.153.130:37611)
I20260810 18:53:39.501747 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38969
--builtin_ntp_servers=127.30.153.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 18:53:39.616515 6423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:39.616837 6423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:39.616911 6423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:39.621254 6423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:39.621335 6423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:39.621419 6423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:39.626168 6423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:39.627326 6423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:39.628950 6423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:39.639309 6431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:39.639441 6434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:39.639501 6423 server_base.cc:1034] running on GCE node
W20260810 18:53:39.639353 6430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:39.640036 6423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:39.640800 6423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:39.641983 6423 hybrid_clock.cc:648] HybridClock initialized: now 1786388019641946 us; error 50 us; skew 500 ppm
I20260810 18:53:39.644184 6423 webserver.cc:460] Webserver started at http://127.30.153.131:45087/ using document root <none> and password file <none>
I20260810 18:53:39.644789 6423 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:39.644856 6423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:39.645107 6423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:39.647526 6423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "8a2927fb21cb7e926dad64abc81c90a0"
server_key_iv: "8d368bfb8c546b6b95fb8745dca48385"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.648329 6423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "8a2927fb21cb7e926dad64abc81c90a0"
server_key_iv: "8d368bfb8c546b6b95fb8745dca48385"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.652763 6423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:39.655808 6442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.657374 6423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:53:39.657512 6423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b"
format_stamp: "Formatted at 2026-08-10 18:53:39 on dist-test-slave-68xk"
server_key: "8a2927fb21cb7e926dad64abc81c90a0"
server_key_iv: "8d368bfb8c546b6b95fb8745dca48385"
server_key_version: "encryptionkey@0"
I20260810 18:53:39.657656 6423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:39.662993 6423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:39.663672 6423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:39.663864 6423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:39.664453 6423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:39.665512 6423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:39.665562 6423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.665632 6423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:39.665666 6423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:39.675678 6423 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:40863
I20260810 18:53:39.675866 6604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:40863 every 8 connection(s)
I20260810 18:53:39.676849 6423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:39.679028 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 6423
I20260810 18:53:39.679221 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:39.679605 31334 external_mini_cluster.cc:1428] Setting key a0030dd10be154b847874e81e236ba8a
I20260810 18:53:39.684700 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.679039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:46607 (local address 127.30.153.190:38969)
0810 18:53:39.679253 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.679258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.680554 (+ 1296us) server_negotiation.cc:407] Connection header received
0810 18:53:39.682235 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.682238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.682288 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.682393 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.684252 (+ 1859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.684256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.684258 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.684271 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.684294 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.684303 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.684388 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.684534 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.684576 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:39.685700 6609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.679604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38969 (local address 127.30.153.131:46607)
0810 18:53:39.680386 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.680443 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:39.682002 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:39.682607 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:39.682617 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:39.682716 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:39.683963 (+ 1247us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:39.683965 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:39.684008 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:39.684081 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:39.684391 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:39.684399 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:39.684403 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:53:39.684547 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 18:53:39.684765 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":141,"threads_started":1}
I20260810 18:53:39.687071 6607 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38969
I20260810 18:53:39.687330 6607 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:39.687822 6607 heartbeater.cc:511] Master 127.30.153.190:38969 requested a full tablet report, sending...
I20260810 18:53:39.688983 5972 ts_manager.cc:194] Registered new tserver with Master: cb3f7f88b1cb4c3ba9e42a315d343f8b (127.30.153.131:40863)
I20260810 18:53:39.694662 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:39.698868 6029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.696944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.30.153.190:38969)
0810 18:53:39.697124 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.697128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.697255 (+ 127us) server_negotiation.cc:407] Connection header received
0810 18:53:39.697370 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.697372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.697413 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.697497 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.698370 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.698373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.698375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.698395 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.698443 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.698452 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.698530 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.698644 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.698684 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:53:39.702379 5972 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:39.705825 5972 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:39.719041 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.715402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:42391 (local address 127.0.0.1:53510)
0810 18:53:39.716045 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.716104 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:39.716223 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:39.717223 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:39.717230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:39.717311 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:39.718112 (+ 801us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:39.718114 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:39.718123 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:39.718142 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:39.718650 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:39.718653 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:39.718656 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 18:53:39.718778 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 18:53:39.718881 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":22,"spinlock_wait_cycles":8192,"thread_start_us":131,"threads_started":1}
I20260810 18:53:39.719189 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.716218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51566 (local address 127.30.153.131:40863)
0810 18:53:39.716640 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.716667 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.716695 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:53:39.716757 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.716763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.716965 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.717096 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.718286 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.718297 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.718312 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.718328 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.718376 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.718438 (+ 62us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.718549 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.718961 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.719029 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":177,"threads_started":1}
I20260810 18:53:39.719369 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.715999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:40863 (local address 127.0.0.1:51566)
0810 18:53:39.716311 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.716326 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:39.716417 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:39.717223 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:39.717230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:39.717311 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:39.718008 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:39.718010 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:39.718034 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:39.718122 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:39.718629 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:39.718640 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:39.718646 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:53:39.718778 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 18:53:39.718869 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":96,"threads_started":1}
I20260810 18:53:39.719072 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.715614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53510 (local address 127.30.153.129:42391)
0810 18:53:39.716750 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.716766 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.716791 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:53:39.716848 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.716854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.717038 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.717766 (+ 728us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.718438 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.718443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.718456 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.718470 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.718518 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.718548 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.718667 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.718804 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.718881 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":131,"threads_started":1}
I20260810 18:53:39.719939 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.715186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:37611 (local address 127.0.0.1:38076)
0810 18:53:39.716045 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.716104 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:39.716223 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:39.718074 (+ 1851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:39.718079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:39.718101 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:39.718603 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:39.718606 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:39.718620 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:39.718649 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:39.719700 (+ 1051us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:39.719704 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:39.719709 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:53:39.719783 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 18:53:39.719818 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"spinlock_wait_cycles":3840,"thread_start_us":97,"threads_started":1}
I20260810 18:53:39.720492 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.715633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.30.153.130:37611)
0810 18:53:39.716700 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.716712 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.716733 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:53:39.716804 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.716809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.717901 (+ 1092us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.718339 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.719468 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.719476 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.719489 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.719502 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.719555 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.719581 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.720221 (+ 640us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.720270 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.720339 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":968,"thread_start_us":127,"threads_started":1}
I20260810 18:53:39.722627 6514 tablet_service.cc:1467] Processing CreateTablet for tablet b338b114634e4ef1b34b82f8e2608646 (DEFAULT_TABLE table=test-table [id=7def0558fbef4e6bac8603e5e2afec68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:39.722631 6142 tablet_service.cc:1467] Processing CreateTablet for tablet b338b114634e4ef1b34b82f8e2608646 (DEFAULT_TABLE table=test-table [id=7def0558fbef4e6bac8603e5e2afec68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:39.722855 6330 tablet_service.cc:1467] Processing CreateTablet for tablet b338b114634e4ef1b34b82f8e2608646 (DEFAULT_TABLE table=test-table [id=7def0558fbef4e6bac8603e5e2afec68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:39.723809 6142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b338b114634e4ef1b34b82f8e2608646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:39.723809 6514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b338b114634e4ef1b34b82f8e2608646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:39.724021 6330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b338b114634e4ef1b34b82f8e2608646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:39.730291 6631 tablet_bootstrap.cc:492] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9: Bootstrap starting.
I20260810 18:53:39.731016 6634 tablet_bootstrap.cc:492] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b: Bootstrap starting.
I20260810 18:53:39.731642 6632 tablet_bootstrap.cc:492] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0: Bootstrap starting.
I20260810 18:53:39.733486 6631 tablet_bootstrap.cc:654] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:39.733866 6632 tablet_bootstrap.cc:654] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:39.733866 6634 tablet_bootstrap.cc:654] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:39.734313 6631 log.cc:824] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:39.734737 6632 log.cc:824] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:39.734738 6634 log.cc:824] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:39.736378 6631 tablet_bootstrap.cc:492] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9: No bootstrap required, opened a new log
I20260810 18:53:39.736557 6634 tablet_bootstrap.cc:492] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b: No bootstrap required, opened a new log
I20260810 18:53:39.736601 6631 ts_tablet_manager.cc:1397] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260810 18:53:39.736720 6632 tablet_bootstrap.cc:492] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0: No bootstrap required, opened a new log
I20260810 18:53:39.736801 6634 ts_tablet_manager.cc:1397] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:53:39.736907 6632 ts_tablet_manager.cc:1397] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:53:39.740442 6634 raft_consensus.cc:348] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.740679 6634 raft_consensus.cc:374] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:39.740733 6634 raft_consensus.cc:729] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3f7f88b1cb4c3ba9e42a315d343f8b, State: Initialized, Role: FOLLOWER
I20260810 18:53:39.741200 6634 consensus_queue.cc:260] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.741299 6631 raft_consensus.cc:348] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.741299 6632 raft_consensus.cc:348] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.741547 6631 raft_consensus.cc:374] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:39.741546 6632 raft_consensus.cc:374] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:39.741603 6632 raft_consensus.cc:729] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faef10ffb9cb47ff9c8779acd31cd5a0, State: Initialized, Role: FOLLOWER
I20260810 18:53:39.741602 6631 raft_consensus.cc:729] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a7bf51f87774a079fcb773a873256b9, State: Initialized, Role: FOLLOWER
I20260810 18:53:39.742043 6631 consensus_queue.cc:260] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.742043 6632 consensus_queue.cc:260] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.742249 6634 ts_tablet_manager.cc:1428] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260810 18:53:39.742662 6607 heartbeater.cc:503] Master 127.30.153.190:38969 was elected leader, sending a full tablet report...
I20260810 18:53:39.742980 6420 heartbeater.cc:503] Master 127.30.153.190:38969 was elected leader, sending a full tablet report...
I20260810 18:53:39.743000 6632 ts_tablet_manager.cc:1428] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:53:39.743191 6236 heartbeater.cc:503] Master 127.30.153.190:38969 was elected leader, sending a full tablet report...
I20260810 18:53:39.744066 6631 ts_tablet_manager.cc:1428] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260810 18:53:39.801743 6641 raft_consensus.cc:482] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:39.802018 6641 raft_consensus.cc:504] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.803347 6641 leader_election.cc:288] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers faef10ffb9cb47ff9c8779acd31cd5a0 (127.30.153.129:42391), cb3f7f88b1cb4c3ba9e42a315d343f8b (127.30.153.131:40863)
I20260810 18:53:39.805670 6422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.803682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:40863 (local address 127.30.153.130:32883)
0810 18:53:39.803943 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.803961 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:39.804065 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:39.804446 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:39.804452 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:39.804469 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:39.805018 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:39.805020 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:39.805039 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:39.805064 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:39.805396 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:39.805399 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:39.805401 (+ 2us) client_negotiation.cc:769] Sending connection context
0810 18:53:39.805463 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 18:53:39.805519 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260810 18:53:39.805694 6630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.804018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:32883 (local address 127.30.153.131:40863)
0810 18:53:39.804183 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.804190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.804211 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:53:39.804257 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.804259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.804291 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.804398 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.805233 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.805242 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.805245 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.805257 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.805278 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.805287 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.805406 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.805480 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.805532 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 18:53:39.805800 6624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.803828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:40575 (local address 127.30.153.129:42391)
0810 18:53:39.803999 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.804006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.804371 (+ 365us) server_negotiation.cc:407] Connection header received
0810 18:53:39.804493 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.804503 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.804539 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.804610 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.805386 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.805389 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.805392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.805406 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.805428 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.805442 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.805540 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.805624 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.805664 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:53:39.805824 6642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.803682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:42391 (local address 127.30.153.130:40575)
0810 18:53:39.804266 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.804278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:39.804391 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:39.804667 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:39.804672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:39.804692 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:39.805225 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:53:39.805227 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:39.805242 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:53:39.805263 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:39.805534 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:39.805536 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:39.805539 (+ 3us) client_negotiation.cc:769] Sending connection context
0810 18:53:39.805609 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 18:53:39.805637 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":172,"thread_start_us":139,"threads_started":1}
W20260810 18:53:39.806155 6238 tablet.cc:2369] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:39.806453 6544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b338b114634e4ef1b34b82f8e2608646" candidate_uuid: "6a7bf51f87774a079fcb773a873256b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" is_pre_election: true
I20260810 18:53:39.806463 6168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b338b114634e4ef1b34b82f8e2608646" candidate_uuid: "6a7bf51f87774a079fcb773a873256b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" is_pre_election: true
I20260810 18:53:39.806779 6168 raft_consensus.cc:2454] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a7bf51f87774a079fcb773a873256b9 in term 0.
I20260810 18:53:39.806805 6544 raft_consensus.cc:2454] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a7bf51f87774a079fcb773a873256b9 in term 0.
I20260810 18:53:39.807345 6263 leader_election.cc:302] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a7bf51f87774a079fcb773a873256b9, faef10ffb9cb47ff9c8779acd31cd5a0; no voters:
I20260810 18:53:39.807652 6641 raft_consensus.cc:2781] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:39.807765 6641 raft_consensus.cc:482] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:39.807827 6641 raft_consensus.cc:3037] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:39.870635 6641 raft_consensus.cc:504] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.871253 6641 leader_election.cc:288] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [CANDIDATE]: Term 1 election: Requested vote from peers faef10ffb9cb47ff9c8779acd31cd5a0 (127.30.153.129:42391), cb3f7f88b1cb4c3ba9e42a315d343f8b (127.30.153.131:40863)
I20260810 18:53:39.871819 6544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b338b114634e4ef1b34b82f8e2608646" candidate_uuid: "6a7bf51f87774a079fcb773a873256b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b"
I20260810 18:53:39.871902 6168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b338b114634e4ef1b34b82f8e2608646" candidate_uuid: "6a7bf51f87774a079fcb773a873256b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0"
I20260810 18:53:39.872004 6544 raft_consensus.cc:3037] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:39.872073 6168 raft_consensus.cc:3037] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:39.873279 6544 raft_consensus.cc:2454] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a7bf51f87774a079fcb773a873256b9 in term 1.
I20260810 18:53:39.873684 6261 leader_election.cc:302] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a7bf51f87774a079fcb773a873256b9, cb3f7f88b1cb4c3ba9e42a315d343f8b; no voters:
I20260810 18:53:39.873854 6168 raft_consensus.cc:2454] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a7bf51f87774a079fcb773a873256b9 in term 1.
I20260810 18:53:39.873890 6641 raft_consensus.cc:2781] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:39.874269 6641 raft_consensus.cc:686] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 1 LEADER]: Becoming Leader. State: Replica: 6a7bf51f87774a079fcb773a873256b9, State: Running, Role: LEADER
I20260810 18:53:39.874722 6641 consensus_queue.cc:237] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } }
I20260810 18:53:39.878578 5975 catalog_manager.cc:5495] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a7bf51f87774a079fcb773a873256b9 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "6a7bf51f87774a079fcb773a873256b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a7bf51f87774a079fcb773a873256b9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 37611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:39.910288 6627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.908575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.30.153.130:37611)
0810 18:53:39.908736 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:39.908743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:39.908762 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:53:39.908863 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:39.908865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:39.908927 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:39.909025 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:39.909839 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:39.909843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:39.909845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:39.909857 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:39.909885 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:39.909893 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:39.909996 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:39.910083 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 18:53:39.910126 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:53:39.916570 6168 raft_consensus.cc:1261] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [term 1 FOLLOWER]: Refusing update from remote peer 6a7bf51f87774a079fcb773a873256b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:39.916569 6544 raft_consensus.cc:1261] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 1 FOLLOWER]: Refusing update from remote peer 6a7bf51f87774a079fcb773a873256b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:39.917526 6643 consensus_queue.cc:1035] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faef10ffb9cb47ff9c8779acd31cd5a0" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:39.917799 6641 consensus_queue.cc:1035] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb3f7f88b1cb4c3ba9e42a315d343f8b" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:39.927129 6649 mvcc.cc:205] Tried to move back new op lower bound from 7317045329569853440 to 7317045329409900544. Current Snapshot: MvccSnapshot[applied={T|T < 7317045329569853440}]
I20260810 18:53:39.936291 5972 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:39.936481 5972 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:39.939091 5972 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd2105d9958548debafa45ca1bda596a: Sending DeleteTablet for 3 replicas of tablet b338b114634e4ef1b34b82f8e2608646
I20260810 18:53:39.939945 6330 tablet_service.cc:1514] Processing DeleteTablet for tablet b338b114634e4ef1b34b82f8e2608646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:39 UTC) from {username='slave'} at 127.0.0.1:38076
I20260810 18:53:39.939968 6142 tablet_service.cc:1514] Processing DeleteTablet for tablet b338b114634e4ef1b34b82f8e2608646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:39 UTC) from {username='slave'} at 127.0.0.1:53510
I20260810 18:53:39.939965 6514 tablet_service.cc:1514] Processing DeleteTablet for tablet b338b114634e4ef1b34b82f8e2608646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:39 UTC) from {username='slave'} at 127.0.0.1:51566
I20260810 18:53:39.940511 6663 tablet_replica.cc:330] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b: stopping tablet replica
I20260810 18:53:39.940600 6664 tablet_replica.cc:330] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9: stopping tablet replica
I20260810 18:53:39.940835 6661 tablet_replica.cc:330] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0: stopping tablet replica
I20260810 18:53:39.940943 6664 raft_consensus.cc:2229] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:39.941267 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 6056
I20260810 18:53:39.941349 6664 raft_consensus.cc:2258] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:39.941639 6661 raft_consensus.cc:2229] T b338b114634e4ef1b34b82f8e2608646 P faef10ffb9cb47ff9c8779acd31cd5a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:39.941282 6663 raft_consensus.cc:2229] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:39.942972 6664 ts_tablet_manager.cc:1905] T b338b114634e4ef1b34b82f8e2608646 P 6a7bf51f87774a079fcb773a873256b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:39.944445 6663 raft_consensus.cc:2258] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:39.957599 6663 ts_tablet_manager.cc:1905] T b338b114634e4ef1b34b82f8e2608646 P cb3f7f88b1cb4c3ba9e42a315d343f8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:39.962968 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 6241
W20260810 18:53:39.963011 5950 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:42391 (error 108)
I20260810 18:53:39.972298 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.971888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:42391 (local address 127.0.0.1:53512)
0810 18:53:39.972083 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.972211 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:42391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260810 18:53:39.972509 5950 catalog_manager.cc:4584] TS faef10ffb9cb47ff9c8779acd31cd5a0 (127.30.153.129:42391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b338b114634e4ef1b34b82f8e2608646: Network error: Client connection negotiation failed: client connection to 127.30.153.129:42391: connect: Connection refused (error 111)
I20260810 18:53:39.981058 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 6423
I20260810 18:53:39.981426 6623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:39.981117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:37611 (local address 127.0.0.1:38098)
0810 18:53:39.981260 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:39.981328 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.130:37611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 18:53:39.991838 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 5931
2026-08-10T18:53:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1252 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 18:53:40.020840 31334 mini_oidc.cc:152] Starting JWKS server
I20260810 18:53:40.022718 31334 webserver.cc:460] Webserver started at https://127.0.0.1:42365/ using document root <none> and password file <none>
I20260810 18:53:40.022927 31334 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:53:40.023242 31334 webserver.cc:460] Webserver started at http://127.0.0.1:38563/ using document root <none> and password file <none>
2026-08-10T18:53:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:40Z Disabled control of system clock
I20260810 18:53:40.034138 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:35571
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:33633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:35571
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42365/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/testchainca.cert with env {}
W20260810 18:53:40.148674 6674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:40.148984 6674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:40.149080 6674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:40.153183 6674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:40.153249 6674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:40.153264 6674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:40.153281 6674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:40.153296 6674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:53:40.153312 6674 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:53:40.153326 6674 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42365/jwks.json
I20260810 18:53:40.158808 6674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:35571
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:35571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42365/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:40.159926 6674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:40.161033 6674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:40.169601 6680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:40.169695 6684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:40.169881 6674 server_base.cc:1034] running on GCE node
W20260810 18:53:40.169632 6681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:40.170590 6674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:40.171667 6674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:40.172834 6674 hybrid_clock.cc:648] HybridClock initialized: now 1786388020172803 us; error 46 us; skew 500 ppm
I20260810 18:53:40.174832 6674 webserver.cc:460] Webserver started at http://127.30.153.190:39985/ using document root <none> and password file <none>
I20260810 18:53:40.175385 6674 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:40.175446 6674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:40.175604 6674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:40.177651 6674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "e09317bc03e14db9adae6a97d10cb6b4"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "278834cc51641c8376e1d4fae4f9c3ac"
server_key_iv: "80aae435fbf46b5a328b349385c60fa4"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.178164 6674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "e09317bc03e14db9adae6a97d10cb6b4"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "278834cc51641c8376e1d4fae4f9c3ac"
server_key_iv: "80aae435fbf46b5a328b349385c60fa4"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.181896 6674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:40.184649 6690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:40.186242 6674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:40.186362 6674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "e09317bc03e14db9adae6a97d10cb6b4"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "278834cc51641c8376e1d4fae4f9c3ac"
server_key_iv: "80aae435fbf46b5a328b349385c60fa4"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.186538 6674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:40.205214 6674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:40.205921 6674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:40.206085 6674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:40.213905 6674 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:35571
I20260810 18:53:40.213932 6765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:35571 every 8 connection(s)
I20260810 18:53:40.214962 6674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:40.218631 6767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:40.220517 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 6674
I20260810 18:53:40.220695 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:40.221050 31334 external_mini_cluster.cc:1428] Setting key 0da21ee67b4e36a95ccbfed0ced3e986
I20260810 18:53:40.226233 6767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4: Bootstrap starting.
I20260810 18:53:40.228834 6767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:40.229841 6767 log.cc:824] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:40.231660 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:40.222710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39120 (local address 127.30.153.190:35571)
0810 18:53:40.223557 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:40.223577 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:40.223626 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:53:40.224446 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:40.224467 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:40.224922 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:40.225311 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:40.225809 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.226963 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.227982 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.228497 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.228696 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.229709 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:40.229714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:40.229727 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:40.230214 (+ 487us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:40.230276 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:40.230299 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:40.230420 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:40.230629 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 18:53:40.230899 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":168,"threads_started":1}
I20260810 18:53:40.232241 6767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4: No bootstrap required, opened a new log
I20260810 18:53:40.234688 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:40.236740 6767 raft_consensus.cc:348] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09317bc03e14db9adae6a97d10cb6b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35571 } }
I20260810 18:53:40.236974 6767 raft_consensus.cc:374] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:40.237012 6767 raft_consensus.cc:729] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e09317bc03e14db9adae6a97d10cb6b4, State: Initialized, Role: FOLLOWER
I20260810 18:53:40.237462 6767 consensus_queue.cc:260] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09317bc03e14db9adae6a97d10cb6b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35571 } }
I20260810 18:53:40.237609 6767 raft_consensus.cc:388] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:40.237663 6767 raft_consensus.cc:482] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:40.237743 6767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:40.238783 6767 raft_consensus.cc:504] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09317bc03e14db9adae6a97d10cb6b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35571 } }
I20260810 18:53:40.239111 6767 leader_election.cc:302] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e09317bc03e14db9adae6a97d10cb6b4; no voters:
I20260810 18:53:40.239392 6767 leader_election.cc:288] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:40.239549 6772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:40.239828 6772 raft_consensus.cc:686] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [term 1 LEADER]: Becoming Leader. State: Replica: e09317bc03e14db9adae6a97d10cb6b4, State: Running, Role: LEADER
I20260810 18:53:40.240159 6772 consensus_queue.cc:237] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09317bc03e14db9adae6a97d10cb6b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35571 } }
I20260810 18:53:40.240918 6767 sys_catalog.cc:564] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:40.242287 6774 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e09317bc03e14db9adae6a97d10cb6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09317bc03e14db9adae6a97d10cb6b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35571 } } }
I20260810 18:53:40.242290 6775 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e09317bc03e14db9adae6a97d10cb6b4. Latest consensus state: current_term: 1 leader_uuid: "e09317bc03e14db9adae6a97d10cb6b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09317bc03e14db9adae6a97d10cb6b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35571 } } }
I20260810 18:53:40.242508 6775 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:40.242771 6774 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:40.243570 6784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:40.246683 6784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:40.252681 6784 catalog_manager.cc:1347] Generated new cluster ID: 2451226b6f4c4be0aa2df7c0137b3dbb
I20260810 18:53:40.252768 6784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:40.287714 6784 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:40.288883 6784 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:40.297329 6784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e09317bc03e14db9adae6a97d10cb6b4: Generated new TSK 0
I20260810 18:53:40.298130 6784 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 18:53:40.325346 6805 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 18:53:40.325551 6805 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:53:40.326074 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:40.308813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39136 (local address 127.30.153.190:35571)
0810 18:53:40.308966 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:40.308971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:40.309262 (+ 291us) server_negotiation.cc:407] Connection header received
0810 18:53:40.309481 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:40.309484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:40.309534 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:40.309613 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:53:40.309904 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.310512 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.311783 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.312008 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.312086 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.312478 (+ 392us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:53:40.325715 (+ 13237us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 18:53:40.325963 (+ 248us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39136: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 18:53:40.326182 6770 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39136: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 18:53:40.327746 6804 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:40.308755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:35571 (local address 127.0.0.1:39136)
0810 18:53:40.309139 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:40.309156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:40.309298 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:40.309658 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:40.309661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:40.309670 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:53:40.309794 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:40.309801 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.310644 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.310649 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:40.311659 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:40.311670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.312130 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.312134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:40.312235 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.312289 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:53:40.325944 (+ 13655us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 18:53:40.326204 (+ 260us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.153.190:35571: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":136,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.153.190:35571: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42365/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 18:53:40.328657 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 6674
2026-08-10T18:53:40Z chronyd exiting
I20260810 18:53:40.376104 31334 test_util.cc:175] -----------------------------------------------
I20260810 18:53:40.376204 31334 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786387990686100-31334-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (357 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 18:53:40.377424 31334 mini_oidc.cc:152] Starting JWKS server
I20260810 18:53:40.378345 31334 webserver.cc:460] Webserver started at https://127.0.0.1:44893/ using document root <none> and password file <none>
I20260810 18:53:40.378504 31334 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:53:40.378813 31334 webserver.cc:460] Webserver started at http://127.0.0.1:34175/ using document root <none> and password file <none>
2026-08-10T18:53:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:40Z Disabled control of system clock
I20260810 18:53:40.389547 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:35445
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:40767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:35445
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44893/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/testchainca.cert with env {}
W20260810 18:53:40.502278 6815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:40.502642 6815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:40.502727 6815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:40.506696 6815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:40.506795 6815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:40.506824 6815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:40.506865 6815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:40.506896 6815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:53:40.506932 6815 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:53:40.506963 6815 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44893/jwks.json
I20260810 18:53:40.512243 6815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:40767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:35445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44893/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:40.513396 6815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:40.514444 6815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:40.523036 6822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:40.523043 6826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:40.523172 6824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:40.523567 6815 server_base.cc:1034] running on GCE node
I20260810 18:53:40.524147 6815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:40.525303 6815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:40.526486 6815 hybrid_clock.cc:648] HybridClock initialized: now 1786388020526467 us; error 31 us; skew 500 ppm
I20260810 18:53:40.528419 6815 webserver.cc:460] Webserver started at http://127.30.153.190:37623/ using document root <none> and password file <none>
I20260810 18:53:40.529018 6815 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:40.529083 6815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:40.529297 6815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:40.531416 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "57aedef20bfc45cd8b0194cb881ff0da"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "6ae0b263003b3e2300aed142469b7a39"
server_key_iv: "a5c9ee660f216e05e6a3107dfa86df75"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.532039 6815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "57aedef20bfc45cd8b0194cb881ff0da"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "6ae0b263003b3e2300aed142469b7a39"
server_key_iv: "a5c9ee660f216e05e6a3107dfa86df75"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.535871 6815 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:53:40.538376 6834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:40.539958 6815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:40.540112 6815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "57aedef20bfc45cd8b0194cb881ff0da"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "6ae0b263003b3e2300aed142469b7a39"
server_key_iv: "a5c9ee660f216e05e6a3107dfa86df75"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.540230 6815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:40.553664 6815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:40.554327 6815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:40.554579 6815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:40.561580 6815 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:35445
I20260810 18:53:40.561590 6907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:35445 every 8 connection(s)
I20260810 18:53:40.562568 6815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:40.565627 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 6815
I20260810 18:53:40.565764 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:40.566074 31334 external_mini_cluster.cc:1428] Setting key 40ca98492a1114092a84fb686cb15013
I20260810 18:53:40.566088 6909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:40.572517 6909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da: Bootstrap starting.
I20260810 18:53:40.574870 6909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:40.575730 6909 log.cc:824] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:40.575982 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:40.567584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33500 (local address 127.30.153.190:35445)
0810 18:53:40.568279 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:40.568295 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:40.568331 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:53:40.569270 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:40.569291 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:40.569676 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:40.570025 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:40.570541 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.571501 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.572446 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.572994 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.573177 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.574143 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:40.574148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:40.574161 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:40.574628 (+ 467us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:40.574687 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:40.574710 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:40.574823 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:40.574946 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 18:53:40.575215 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":157,"threads_started":1}
I20260810 18:53:40.578794 6909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da: No bootstrap required, opened a new log
I20260810 18:53:40.579003 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:40.582940 6909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aedef20bfc45cd8b0194cb881ff0da" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35445 } }
I20260810 18:53:40.583214 6909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:40.583302 6909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57aedef20bfc45cd8b0194cb881ff0da, State: Initialized, Role: FOLLOWER
I20260810 18:53:40.583782 6909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aedef20bfc45cd8b0194cb881ff0da" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35445 } }
I20260810 18:53:40.583940 6909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:40.584026 6909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:40.584133 6909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:40.585175 6909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aedef20bfc45cd8b0194cb881ff0da" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35445 } }
I20260810 18:53:40.585587 6909 leader_election.cc:302] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57aedef20bfc45cd8b0194cb881ff0da; no voters:
I20260810 18:53:40.586040 6909 leader_election.cc:288] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:40.586194 6914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:40.586488 6914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [term 1 LEADER]: Becoming Leader. State: Replica: 57aedef20bfc45cd8b0194cb881ff0da, State: Running, Role: LEADER
I20260810 18:53:40.586819 6914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aedef20bfc45cd8b0194cb881ff0da" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35445 } }
I20260810 18:53:40.587405 6909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:40.588771 6915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57aedef20bfc45cd8b0194cb881ff0da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aedef20bfc45cd8b0194cb881ff0da" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35445 } } }
I20260810 18:53:40.588762 6916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57aedef20bfc45cd8b0194cb881ff0da. Latest consensus state: current_term: 1 leader_uuid: "57aedef20bfc45cd8b0194cb881ff0da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57aedef20bfc45cd8b0194cb881ff0da" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35445 } } }
I20260810 18:53:40.588935 6915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:40.588935 6916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:40.589447 6925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:40.592728 6925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:40.598719 6925 catalog_manager.cc:1347] Generated new cluster ID: 31709a58474f4c4b946fd77a42b82e0e
I20260810 18:53:40.598803 6925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:40.623629 6925 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:40.624946 6925 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:40.639202 6925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57aedef20bfc45cd8b0194cb881ff0da: Generated new TSK 0
I20260810 18:53:40.640206 6925 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 18:53:40.679939 6945 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 18:53:40.680047 6945 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 18:53:40.680706 6912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:40.657220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.30.153.190:35445)
0810 18:53:40.657377 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:40.657384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:40.657580 (+ 196us) server_negotiation.cc:407] Connection header received
0810 18:53:40.657762 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:40.657765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:40.657827 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:40.657917 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:53:40.658194 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.658872 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.660154 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.660387 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.660469 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.660811 (+ 342us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:53:40.680323 (+ 19512us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 18:53:40.680586 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33516: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 18:53:40.680799 6912 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33516: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 18:53:40.680958 6944 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:40.657125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:35445 (local address 127.0.0.1:33516)
0810 18:53:40.657482 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:40.657498 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:40.657617 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:40.657952 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:40.657955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:40.657964 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:53:40.658083 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:40.658090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.659005 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.659009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:40.660021 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:40.660032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.660510 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.660514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:40.660606 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.660637 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:53:40.680550 (+ 19913us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 18:53:40.680770 (+ 220us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.153.190:35445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":119,"threads_started":1}
I20260810 18:53:40.681675 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 6815
2026-08-10T18:53:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 18:53:40.729729 31334 mini_oidc.cc:152] Starting JWKS server
I20260810 18:53:40.730659 31334 webserver.cc:460] Webserver started at https://127.0.0.1:42965/ using document root <none> and password file <none>
I20260810 18:53:40.730751 31334 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 18:53:40.730957 31334 webserver.cc:460] Webserver started at http://127.0.0.1:42999/ using document root <none> and password file <none>
2026-08-10T18:53:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:40Z Disabled control of system clock
I20260810 18:53:40.741600 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:44927
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:36359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:44927
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42965/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 18:53:40.857437 6953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:40.857740 6953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:40.857779 6953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:40.861927 6953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:40.861990 6953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:40.862006 6953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:40.862023 6953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:40.862038 6953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:53:40.862054 6953 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:53:40.862068 6953 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42965/jwks.json
I20260810 18:53:40.866869 6953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:44927
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:44927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42965/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:40.867913 6953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:40.869277 6953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:40.878150 6960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:40.878191 6961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:40.878218 6963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:40.878291 6953 server_base.cc:1034] running on GCE node
I20260810 18:53:40.878961 6953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:40.880113 6953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:40.881350 6953 hybrid_clock.cc:648] HybridClock initialized: now 1786388020881318 us; error 43 us; skew 500 ppm
I20260810 18:53:40.888736 6953 webserver.cc:460] Webserver started at http://127.30.153.190:46639/ using document root <none> and password file <none>
I20260810 18:53:40.889385 6953 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:40.889488 6953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:40.889740 6953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:40.891953 6953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "9f08994387e14ac28db70074abb5e715"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "e078fd4fa91fec2269193340db287bf6"
server_key_iv: "c7c466d1afa130c17713bea59ba937c3"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.892616 6953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "9f08994387e14ac28db70074abb5e715"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "e078fd4fa91fec2269193340db287bf6"
server_key_iv: "c7c466d1afa130c17713bea59ba937c3"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.896632 6953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:40.899210 6971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:40.900594 6953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:53:40.900744 6953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "9f08994387e14ac28db70074abb5e715"
format_stamp: "Formatted at 2026-08-10 18:53:40 on dist-test-slave-68xk"
server_key: "e078fd4fa91fec2269193340db287bf6"
server_key_iv: "c7c466d1afa130c17713bea59ba937c3"
server_key_version: "encryptionkey@0"
I20260810 18:53:40.900863 6953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:40.931095 6953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:40.931874 6953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:40.932091 6953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:40.939482 6953 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:44927
I20260810 18:53:40.939533 7038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:44927 every 8 connection(s)
I20260810 18:53:40.940469 6953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:40.943902 7039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:40.948186 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 6953
I20260810 18:53:40.948336 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:40.948673 31334 external_mini_cluster.cc:1428] Setting key ca52d7658335c6084333196af10251dc
I20260810 18:53:40.950538 7039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715: Bootstrap starting.
I20260810 18:53:40.952947 7039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:40.953760 7039 log.cc:824] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:40.955736 7039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715: No bootstrap required, opened a new log
I20260810 18:53:40.958532 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:40.950115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49860 (local address 127.30.153.190:44927)
0810 18:53:40.950880 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:40.950896 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:40.950932 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:53:40.951587 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:40.951603 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:40.951985 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:40.952336 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:40.952806 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.953862 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.954849 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:40.955320 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:40.955513 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:40.956474 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:40.956479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:40.956492 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:40.956921 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:40.956975 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:40.956996 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:40.957110 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:40.957273 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 18:53:40.957511 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":156,"threads_started":1}
I20260810 18:53:40.959306 7039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f08994387e14ac28db70074abb5e715" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44927 } }
I20260810 18:53:40.959579 7039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:40.959673 7039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f08994387e14ac28db70074abb5e715, State: Initialized, Role: FOLLOWER
I20260810 18:53:40.960156 7039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f08994387e14ac28db70074abb5e715" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44927 } }
I20260810 18:53:40.960323 7039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:40.960395 7039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:40.960493 7039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:40.961817 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:40.961817 7039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f08994387e14ac28db70074abb5e715" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44927 } }
I20260810 18:53:40.962179 7039 leader_election.cc:302] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f08994387e14ac28db70074abb5e715; no voters:
I20260810 18:53:40.962600 7039 leader_election.cc:288] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:40.962713 7047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:40.962963 7047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [term 1 LEADER]: Becoming Leader. State: Replica: 9f08994387e14ac28db70074abb5e715, State: Running, Role: LEADER
I20260810 18:53:40.963296 7047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f08994387e14ac28db70074abb5e715" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44927 } }
I20260810 18:53:40.963966 7039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:40.965003 7049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f08994387e14ac28db70074abb5e715. Latest consensus state: current_term: 1 leader_uuid: "9f08994387e14ac28db70074abb5e715" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f08994387e14ac28db70074abb5e715" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44927 } } }
I20260810 18:53:40.965164 7049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:40.965368 7048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f08994387e14ac28db70074abb5e715" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f08994387e14ac28db70074abb5e715" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44927 } } }
I20260810 18:53:40.965464 7048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:40.965811 7060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:40.968696 7060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:40.973951 7060 catalog_manager.cc:1347] Generated new cluster ID: 5c9f1b7bbc5243b68c55f45676682d29
I20260810 18:53:40.974020 7060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:40.997845 7060 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:40.998868 7060 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:41.011605 7060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f08994387e14ac28db70074abb5e715: Generated new TSK 0
I20260810 18:53:41.012888 7060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:41.040405 7045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:41.031811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.30.153.190:44927)
0810 18:53:41.031958 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:41.031964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:41.032176 (+ 212us) server_negotiation.cc:407] Connection header received
0810 18:53:41.032324 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:41.032328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:41.032388 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:41.032519 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:53:41.032835 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.033766 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.034781 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.035245 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.035357 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:41.035658 (+ 301us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:53:41.039977 (+ 4319us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42965/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 18:53:41.040247 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49868: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42965/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 18:53:41.040485 7045 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49868: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42965/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 18:53:41.041908 7079 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:41.031747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:44927 (local address 127.0.0.1:49868)
0810 18:53:41.032082 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:41.032096 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:41.032198 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:41.032560 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:41.032564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:41.032573 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:53:41.032695 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:41.032712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.033904 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.033907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:41.034661 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:41.034671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.035389 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.035392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:41.035455 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:41.035478 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:53:41.040159 (+ 4681us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42965/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 18:53:41.041741 (+ 1582us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.153.190:44927: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42965/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":1356,"thread_start_us":118,"threads_started":1}
I20260810 18:53:41.042637 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 6953
E20260810 18:53:41.061460 7080 webserver.cc:533] Webserver: sslize failed:
E20260810 18:53:41.061544 7080 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-10T18:53:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (363 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T18:53:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:41Z Disabled control of system clock
I20260810 18:53:41.103847 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42253
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:42253
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/testchainca.cert with env {}
W20260810 18:53:41.218611 7084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:41.218905 7084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:41.218947 7084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:41.222945 7084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:41.223006 7084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:41.223021 7084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:41.223038 7084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:41.223054 7084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 18:53:41.223070 7084 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 18:53:41.223084 7084 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 18:53:41.228498 7084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:42253
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:41.229532 7084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:41.230549 7084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:41.239384 7090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:41.239385 7094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:41.239375 7089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:41.239745 7084 server_base.cc:1034] running on GCE node
I20260810 18:53:41.240270 7084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:41.241394 7084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:41.242538 7084 hybrid_clock.cc:648] HybridClock initialized: now 1786388021242506 us; error 47 us; skew 500 ppm
I20260810 18:53:41.244452 7084 webserver.cc:460] Webserver started at http://127.30.153.190:42207/ using document root <none> and password file <none>
I20260810 18:53:41.245007 7084 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:41.245085 7084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:41.245251 7084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:41.247437 7084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "43122468c1e54f9bbde580fbddd7427b"
format_stamp: "Formatted at 2026-08-10 18:53:41 on dist-test-slave-68xk"
server_key: "bb6582de94787a69007071cefbc98a60"
server_key_iv: "cd9fcb8091e389a9c04a9528bd0d902f"
server_key_version: "encryptionkey@0"
I20260810 18:53:41.248018 7084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "43122468c1e54f9bbde580fbddd7427b"
format_stamp: "Formatted at 2026-08-10 18:53:41 on dist-test-slave-68xk"
server_key: "bb6582de94787a69007071cefbc98a60"
server_key_iv: "cd9fcb8091e389a9c04a9528bd0d902f"
server_key_version: "encryptionkey@0"
I20260810 18:53:41.251823 7084 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:53:41.254168 7100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:41.255504 7084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:41.255626 7084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "43122468c1e54f9bbde580fbddd7427b"
format_stamp: "Formatted at 2026-08-10 18:53:41 on dist-test-slave-68xk"
server_key: "bb6582de94787a69007071cefbc98a60"
server_key_iv: "cd9fcb8091e389a9c04a9528bd0d902f"
server_key_version: "encryptionkey@0"
I20260810 18:53:41.255717 7084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:41.268121 7084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:41.268697 7084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:41.268839 7084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:41.276065 7084 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:42253
I20260810 18:53:41.276072 7180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:42253 every 8 connection(s)
I20260810 18:53:41.276997 7084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:41.279824 7181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:41.280052 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 7084
I20260810 18:53:41.280195 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:41.280486 31334 external_mini_cluster.cc:1428] Setting key 914fa8f4be5250432a5a5be4d1e3a04a
I20260810 18:53:41.287186 7181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b: Bootstrap starting.
I20260810 18:53:41.289876 7181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:41.290832 7181 log.cc:824] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:41.290920 7185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:41.281850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.30.153.190:42253)
0810 18:53:41.282532 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:41.282552 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:41.282594 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:53:41.283334 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:41.283358 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:41.283863 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:41.284219 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:41.284643 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.286145 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.287108 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.287784 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.287939 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:41.288803 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:41.288809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:41.288823 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:53:41.289297 (+ 474us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:41.289368 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:53:41.289395 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:41.289492 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:41.289672 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 18:53:41.289958 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":99,"threads_started":1}
I20260810 18:53:41.292943 7181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b: No bootstrap required, opened a new log
I20260810 18:53:41.293830 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:53:41.296838 7181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43122468c1e54f9bbde580fbddd7427b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42253 } }
I20260810 18:53:41.297058 7181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:41.297137 7181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43122468c1e54f9bbde580fbddd7427b, State: Initialized, Role: FOLLOWER
I20260810 18:53:41.297595 7181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43122468c1e54f9bbde580fbddd7427b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42253 } }
I20260810 18:53:41.297746 7181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:41.297837 7181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:41.297958 7181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:41.298972 7181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43122468c1e54f9bbde580fbddd7427b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42253 } }
I20260810 18:53:41.299322 7181 leader_election.cc:302] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43122468c1e54f9bbde580fbddd7427b; no voters:
I20260810 18:53:41.299644 7181 leader_election.cc:288] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:41.299820 7189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:41.300086 7189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [term 1 LEADER]: Becoming Leader. State: Replica: 43122468c1e54f9bbde580fbddd7427b, State: Running, Role: LEADER
I20260810 18:53:41.300470 7189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43122468c1e54f9bbde580fbddd7427b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42253 } }
I20260810 18:53:41.301045 7181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:41.302549 7190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43122468c1e54f9bbde580fbddd7427b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43122468c1e54f9bbde580fbddd7427b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42253 } } }
I20260810 18:53:41.302703 7190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:41.302670 7191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43122468c1e54f9bbde580fbddd7427b. Latest consensus state: current_term: 1 leader_uuid: "43122468c1e54f9bbde580fbddd7427b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43122468c1e54f9bbde580fbddd7427b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42253 } } }
I20260810 18:53:41.302982 7191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:41.303495 7207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:41.306742 7207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:41.312601 7207 catalog_manager.cc:1347] Generated new cluster ID: 1c6a4e5d5b8442bebc6f6595753102f4
I20260810 18:53:41.312669 7207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:41.326683 7207 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:41.327939 7207 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:41.336612 7207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43122468c1e54f9bbde580fbddd7427b: Generated new TSK 0
I20260810 18:53:41.337314 7207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:41.835829 7185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:41.344190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.30.153.190:42253)
0810 18:53:41.344332 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:41.344337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:41.344541 (+ 204us) server_negotiation.cc:407] Connection header received
0810 18:53:41.344700 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:41.344703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:41.344749 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:41.344842 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0810 18:53:41.345119 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.345687 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.346970 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.347230 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.347317 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:41.347692 (+ 375us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 18:53:41.835384 (+487692us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 18:53:41.835699 (+ 315us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38576: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 18:53:41.835945 7185 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38576: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 18:53:41.836179 7220 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:41.344116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42253 (local address 127.0.0.1:38576)
0810 18:53:41.344445 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:41.344461 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:41.344580 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:41.344872 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:41.344875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:41.344883 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0810 18:53:41.344998 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:41.345004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.345811 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.345814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:41.346836 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:41.346849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:41.347369 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:41.347373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:41.347470 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:41.347498 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 18:53:41.835698 (+488200us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 18:53:41.835921 (+ 223us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.153.190:42253: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":125,"threads_started":1}
I20260810 18:53:41.837054 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 7084
2026-08-10T18:53:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (766 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (154 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:42 dist-test-slave-68xk krb5kdc[7239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:42 dist-test-slave-68xk krb5kdc[7239](info): set up 2 sockets
Aug 10 18:53:42 dist-test-slave-68xk krb5kdc[7239](info): commencing operation
krb5kdc: starting...
W20260810 18:53:43.393146 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:43 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:43.533897 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:34375
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:34375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:43.647843 7260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:43.648175 7260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:43.648247 7260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:43.652304 7260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:43.652406 7260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:43.652446 7260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:43.652485 7260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:43.652522 7260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:43.657831 7260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:34375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:34375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:43.659011 7260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:43.660096 7260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:43.669251 7267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:43.669287 7266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:43.669246 7269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:43.669543 7260 server_base.cc:1034] running on GCE node
I20260810 18:53:43.670146 7260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:43.671353 7260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:43.672529 7260 hybrid_clock.cc:648] HybridClock initialized: now 1786388023672512 us; error 34 us; skew 500 ppm
Aug 10 18:53:43 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388023, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:43.675770 7260 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:43.676779 7260 webserver.cc:460] Webserver started at http://127.30.153.190:46835/ using document root <none> and password file <none>
I20260810 18:53:43.677402 7260 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:43.677469 7260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:43.677707 7260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:43.679908 7260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "86552ec5f71f4ddc9a15c00f32eabf52"
format_stamp: "Formatted at 2026-08-10 18:53:43 on dist-test-slave-68xk"
server_key: "a0471131563b8346c4daa05e8a43d047"
server_key_iv: "f97f793b2c2749ea43f846d4f011f194"
server_key_version: "encryptionkey@0"
I20260810 18:53:43.680514 7260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "86552ec5f71f4ddc9a15c00f32eabf52"
format_stamp: "Formatted at 2026-08-10 18:53:43 on dist-test-slave-68xk"
server_key: "a0471131563b8346c4daa05e8a43d047"
server_key_iv: "f97f793b2c2749ea43f846d4f011f194"
server_key_version: "encryptionkey@0"
I20260810 18:53:43.684438 7260 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:53:43.687031 7277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:43.688319 7260 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:43.688462 7260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "86552ec5f71f4ddc9a15c00f32eabf52"
format_stamp: "Formatted at 2026-08-10 18:53:43 on dist-test-slave-68xk"
server_key: "a0471131563b8346c4daa05e8a43d047"
server_key_iv: "f97f793b2c2749ea43f846d4f011f194"
server_key_version: "encryptionkey@0"
I20260810 18:53:43.688568 7260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:43.697901 7260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:43.701306 7260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:43.701529 7260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:43.709292 7260 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:34375
I20260810 18:53:43.709311 7358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:34375 every 8 connection(s)
I20260810 18:53:43.710306 7260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:43.711171 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 7260
I20260810 18:53:43.711298 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:43.711625 31334 external_mini_cluster.cc:1428] Setting key 8a6d3b1b7c11a96ceef08a74a069fa6d
I20260810 18:53:43.713811 7359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:43.720832 7359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52: Bootstrap starting.
Aug 10 18:53:43 dist-test-slave-68xk krb5kdc[7239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:43.723366 7359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:43.724114 7359 log.cc:824] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:43.726168 7359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52: No bootstrap required, opened a new log
I20260810 18:53:43.726799 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:43.713070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.30.153.190:34375)
0810 18:53:43.713546 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:43.713563 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:43.713601 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:53:43.714353 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:43.714373 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:43.714802 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:43.715098 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:43.716056 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:43.717121 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:43.718274 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:43.718840 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:43.718963 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:43.721915 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:43.721920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:43.721933 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:43.721962 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:43.723787 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:43.724264 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:43.724269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:43.724274 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:43.724383 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:43.724690 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:43.724693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:43.724695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:43.725105 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:43.725306 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:43.725625 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 18:53:43.725921 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":160,"threads_started":1}
I20260810 18:53:43.729840 7359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86552ec5f71f4ddc9a15c00f32eabf52" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34375 } }
I20260810 18:53:43.730087 7359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:43.730170 7359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86552ec5f71f4ddc9a15c00f32eabf52, State: Initialized, Role: FOLLOWER
I20260810 18:53:43.730651 7359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86552ec5f71f4ddc9a15c00f32eabf52" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34375 } }
I20260810 18:53:43.730805 7359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:43.730896 7359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:43.730996 7359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:43.732034 7359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86552ec5f71f4ddc9a15c00f32eabf52" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34375 } }
I20260810 18:53:43.732378 7359 leader_election.cc:302] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86552ec5f71f4ddc9a15c00f32eabf52; no voters:
I20260810 18:53:43.732697 7359 leader_election.cc:288] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:43.732874 7366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:43.733100 7366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [term 1 LEADER]: Becoming Leader. State: Replica: 86552ec5f71f4ddc9a15c00f32eabf52, State: Running, Role: LEADER
I20260810 18:53:43.733458 7366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86552ec5f71f4ddc9a15c00f32eabf52" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34375 } }
I20260810 18:53:43.734016 7359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:43.735394 7367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86552ec5f71f4ddc9a15c00f32eabf52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86552ec5f71f4ddc9a15c00f32eabf52" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34375 } } }
I20260810 18:53:43.735371 7368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86552ec5f71f4ddc9a15c00f32eabf52. Latest consensus state: current_term: 1 leader_uuid: "86552ec5f71f4ddc9a15c00f32eabf52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86552ec5f71f4ddc9a15c00f32eabf52" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34375 } } }
I20260810 18:53:43.735538 7367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:43.735538 7368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:43.735945 7380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:43.739188 7380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:43.744769 7380 catalog_manager.cc:1347] Generated new cluster ID: 01dc8acb1e884428af03041d4ea2d69d
I20260810 18:53:43.744850 7380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:43.760684 7380 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:43.761992 7380 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:43.767944 7380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86552ec5f71f4ddc9a15c00f32eabf52: Generated new TSK 0
I20260810 18:53:43.768589 7380 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:43.806775 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34375
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:43.941828 7392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:43.942157 7392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:43.942243 7392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:43.946574 7392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:43.946671 7392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:43.946774 7392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:43.952155 7392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:43.953402 7392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:43.954540 7392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:43.964986 7401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:43.965024 7403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:43.965339 7392 server_base.cc:1034] running on GCE node
W20260810 18:53:43.964978 7400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:43.965955 7392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:43.966785 7392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:43.967983 7392 hybrid_clock.cc:648] HybridClock initialized: now 1786388023967948 us; error 46 us; skew 500 ppm
Aug 10 18:53:43 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388023, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:43.971541 7392 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:43.973212 7392 webserver.cc:460] Webserver started at http://127.30.153.129:34647/ using document root <none> and password file <none>
I20260810 18:53:43.974192 7392 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:43.974316 7392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:43.974632 7392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:43.977072 7392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "056a118d3bb54defbf9db77faf8a42b4"
format_stamp: "Formatted at 2026-08-10 18:53:43 on dist-test-slave-68xk"
server_key: "ee525cf26be04816f03e1f647ac2d566"
server_key_iv: "6a2e66416ab560a2f5b869692a408479"
server_key_version: "encryptionkey@0"
I20260810 18:53:43.977687 7392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "056a118d3bb54defbf9db77faf8a42b4"
format_stamp: "Formatted at 2026-08-10 18:53:43 on dist-test-slave-68xk"
server_key: "ee525cf26be04816f03e1f647ac2d566"
server_key_iv: "6a2e66416ab560a2f5b869692a408479"
server_key_version: "encryptionkey@0"
I20260810 18:53:43.981961 7392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260810 18:53:43.985222 7415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:43.986634 7392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:43.986779 7392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "056a118d3bb54defbf9db77faf8a42b4"
format_stamp: "Formatted at 2026-08-10 18:53:43 on dist-test-slave-68xk"
server_key: "ee525cf26be04816f03e1f647ac2d566"
server_key_iv: "6a2e66416ab560a2f5b869692a408479"
server_key_version: "encryptionkey@0"
I20260810 18:53:43.986898 7392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:44.000702 7392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:44.004190 7392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:44.004436 7392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:44.005178 7392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:44.006155 7392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:44.006242 7392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.006325 7392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:44.006374 7392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.016861 7392 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:45931
I20260810 18:53:44.016884 7565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:45931 every 8 connection(s)
I20260810 18:53:44.017894 7392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:44.026093 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 7392
I20260810 18:53:44.026278 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:44.026741 31334 external_mini_cluster.cc:1428] Setting key c47876d841ca623cda14354e50e8ff4c
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388023, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:44.032969 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.019836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:60189 (local address 127.30.153.190:34375)
0810 18:53:44.019998 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.020002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.020774 (+ 772us) server_negotiation.cc:407] Connection header received
0810 18:53:44.022171 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.022174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.022224 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.022309 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.023663 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.024204 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.025432 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.025700 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.025901 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.028831 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:44.028835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:44.028838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:44.028868 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:44.030662 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.031263 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.031272 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.031273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.031354 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.031738 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.031743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.031745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.031942 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:44.032068 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:44.032627 (+ 559us) server_negotiation.cc:299] Negotiation successful
0810 18:53:44.032774 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:53:44.033905 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.020094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:34375 (local address 127.30.153.129:60189)
0810 18:53:44.020596 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:44.020642 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:44.021955 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:44.022597 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:44.022612 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:44.022987 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:44.023512 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:44.023524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.024333 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.024336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:44.025282 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:44.025292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.025916 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.025919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:44.026100 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.026903 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:44.026921 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:44.028650 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:44.030802 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:44.030808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:44.030824 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:44.031132 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:44.031460 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:44.031463 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:44.031466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:44.031626 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:44.032068 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:44.032077 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:44.032354 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 18:53:44.032571 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 18:53:44.032925 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":124,"threads_started":1}
I20260810 18:53:44.035413 7566 heartbeater.cc:347] Connected to a master server at 127.30.153.190:34375
I20260810 18:53:44.035701 7566 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:44.036356 7566 heartbeater.cc:511] Master 127.30.153.190:34375 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:53:44.038192 7302 ts_manager.cc:194] Registered new tserver with Master: 056a118d3bb54defbf9db77faf8a42b4 (127.30.153.129:45931)
I20260810 18:53:44.039585 7302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:60189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:44.058611 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34375
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:44.176770 7575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:44.177034 7575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:44.177114 7575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:44.181206 7575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:44.181277 7575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:44.181335 7575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:44.187167 7575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:44.188246 7575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:44.189292 7575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:44.199616 7581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:44.199774 7580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:44.199672 7583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:44.199816 7575 server_base.cc:1034] running on GCE node
I20260810 18:53:44.200417 7575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:44.201071 7575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:44.202236 7575 hybrid_clock.cc:648] HybridClock initialized: now 1786388024202213 us; error 35 us; skew 500 ppm
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.205296 7575 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:44.206175 7575 webserver.cc:460] Webserver started at http://127.30.153.130:46537/ using document root <none> and password file <none>
I20260810 18:53:44.206954 7575 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:44.207053 7575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:44.207297 7575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:44.209365 7575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "e1fcfe4d42d6408cb987d5882a97de58"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-68xk"
server_key: "d85a9eb3c8810bc922880242294ba2b0"
server_key_iv: "f73182e33f3a64c66e3963f218a7418e"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.209940 7575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "e1fcfe4d42d6408cb987d5882a97de58"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-68xk"
server_key: "d85a9eb3c8810bc922880242294ba2b0"
server_key_iv: "f73182e33f3a64c66e3963f218a7418e"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.213722 7575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:44.216238 7592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.217470 7575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260810 18:53:44.217608 7575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "e1fcfe4d42d6408cb987d5882a97de58"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-68xk"
server_key: "d85a9eb3c8810bc922880242294ba2b0"
server_key_iv: "f73182e33f3a64c66e3963f218a7418e"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.217718 7575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:44.234760 7575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:44.238102 7575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:44.238314 7575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:44.238955 7575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:44.239795 7575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:44.239868 7575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.239943 7575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:44.239995 7575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.249476 7575 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:33873
I20260810 18:53:44.249507 7752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:33873 every 8 connection(s)
I20260810 18:53:44.250397 7575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:44.256623 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 7575
I20260810 18:53:44.256749 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:44.257038 31334 external_mini_cluster.cc:1428] Setting key f270b499e2ab21e308a228680361889a
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:44.266281 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.252504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:59623 (local address 127.30.153.190:34375)
0810 18:53:44.252643 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.252646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.253510 (+ 864us) server_negotiation.cc:407] Connection header received
0810 18:53:44.255018 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.255022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.255084 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.255188 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.256354 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.257137 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.258758 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.259090 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.259219 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.261952 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:44.261959 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:44.261965 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:44.261998 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:44.263902 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.264458 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.264465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.264470 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.264572 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.264933 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.264938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.264941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.265230 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:44.265362 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:44.265919 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 18:53:44.266060 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:53:44.267221 7756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.252872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:34375 (local address 127.30.153.130:59623)
0810 18:53:44.253358 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:44.253402 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:44.254802 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:44.255371 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:44.255378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:44.255731 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:44.256207 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:44.256220 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.257278 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.257281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:44.258630 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:44.258641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.259262 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.259265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:44.259421 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.260119 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:44.260138 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:44.261793 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:44.264036 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:44.264041 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:44.264061 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:44.264347 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:44.264679 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:44.264682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:44.264683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:44.264826 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:44.265372 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:44.265378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:44.265636 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 18:53:44.265809 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 18:53:44.266204 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":138,"threads_started":1}
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:53:44.268716 7753 heartbeater.cc:347] Connected to a master server at 127.30.153.190:34375
I20260810 18:53:44.268985 7753 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:44.269562 7753 heartbeater.cc:511] Master 127.30.153.190:34375 requested a full tablet report, sending...
I20260810 18:53:44.270746 7302 ts_manager.cc:194] Registered new tserver with Master: e1fcfe4d42d6408cb987d5882a97de58 (127.30.153.130:33873)
I20260810 18:53:44.271409 7302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:59623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:44.290483 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34375
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:35189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:44.406262 7759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:44.406589 7759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:44.406636 7759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:44.410576 7759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:44.410631 7759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:44.410687 7759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:44.416390 7759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:35189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:44.417392 7759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:44.418342 7759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:44.428336 7768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:44.428373 7766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:44.428386 7771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:44.428608 7759 server_base.cc:1034] running on GCE node
I20260810 18:53:44.429100 7759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:44.429780 7759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:44.430948 7759 hybrid_clock.cc:648] HybridClock initialized: now 1786388024430920 us; error 38 us; skew 500 ppm
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.433815 7759 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:44.434800 7759 webserver.cc:460] Webserver started at http://127.30.153.131:34969/ using document root <none> and password file <none>
I20260810 18:53:44.435456 7759 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:44.435559 7759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:44.435818 7759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:44.437913 7759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "a0e107d4db8a4cdebc18fb98b295b751"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-68xk"
server_key: "5f36786df2c8ce271e4a4df100a8a61d"
server_key_iv: "7c11bf093a8d402643e2d28a92f6444f"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.438557 7759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "a0e107d4db8a4cdebc18fb98b295b751"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-68xk"
server_key: "5f36786df2c8ce271e4a4df100a8a61d"
server_key_iv: "7c11bf093a8d402643e2d28a92f6444f"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.442327 7759 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:44.444947 7780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.446282 7759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:44.446465 7759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "a0e107d4db8a4cdebc18fb98b295b751"
format_stamp: "Formatted at 2026-08-10 18:53:44 on dist-test-slave-68xk"
server_key: "5f36786df2c8ce271e4a4df100a8a61d"
server_key_iv: "7c11bf093a8d402643e2d28a92f6444f"
server_key_version: "encryptionkey@0"
I20260810 18:53:44.446593 7759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:44.459789 7759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:44.462781 7759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:44.462990 7759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:44.463598 7759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:44.464437 7759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:44.464510 7759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.464583 7759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:44.464632 7759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:44.473707 7759 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:39327
I20260810 18:53:44.473737 7946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:39327 every 8 connection(s)
I20260810 18:53:44.474692 7759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:44.478298 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 7759
I20260810 18:53:44.478489 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:44.478843 31334 external_mini_cluster.cc:1428] Setting key 751c5247d8e2e40d346067db2a828c37
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:44.488405 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.476485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:39599 (local address 127.30.153.190:34375)
0810 18:53:44.476642 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.476646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.477471 (+ 825us) server_negotiation.cc:407] Connection header received
0810 18:53:44.479091 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.479095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.479156 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.479290 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.480467 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.480998 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.482136 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.482364 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.482492 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.485047 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:44.485050 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:44.485052 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:44.485074 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:44.486385 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.486929 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.486932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.486933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.487008 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.487385 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.487387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.487388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.487553 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:44.487630 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:44.488148 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 18:53:44.488282 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:53:44.489249 7949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.476859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:34375 (local address 127.30.153.131:39599)
0810 18:53:44.477319 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:44.477363 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:44.478849 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:44.479479 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:44.479486 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:44.479849 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:44.480322 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:44.480335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.481126 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.481132 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:44.482003 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:44.482012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.482534 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.482539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:44.482714 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.483326 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:44.483343 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:44.484880 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:44.486543 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:44.486548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:44.486558 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:44.486830 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:44.487149 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:44.487152 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:44.487153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:44.487296 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:44.487671 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:44.487677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:44.487911 (+ 234us) client_negotiation.cc:769] Sending connection context
0810 18:53:44.488087 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 18:53:44.488389 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":118,"threads_started":1}
I20260810 18:53:44.490375 7947 heartbeater.cc:347] Connected to a master server at 127.30.153.190:34375
I20260810 18:53:44.490732 7947 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:44.491281 7947 heartbeater.cc:511] Master 127.30.153.190:34375 requested a full tablet report, sending...
I20260810 18:53:44.492425 7302 ts_manager.cc:194] Registered new tserver with Master: a0e107d4db8a4cdebc18fb98b295b751 (127.30.153.131:39327)
I20260810 18:53:44.492841 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:44.493167 7302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:39599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/test-admin.keytab.
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.511597 31334 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 18:53:44.514684 7956 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:44.513074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.30.153.129:45931)
0810 18:53:44.513450 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.513466 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.513491 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:53:44.513543 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.513549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.513750 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.513867 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.514472 (+ 605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51182: BlockingRecv error: recv got EOF from 127.0.0.1:51182 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":144,"threads_started":1}
W20260810 18:53:44.515162 31334 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5cc)
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.516228 31334 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.517239 31334 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:44.528497 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.518175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51196 (local address 127.30.153.129:45931)
0810 18:53:44.518324 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.518328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.518341 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:44.518514 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.518517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.518565 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.518643 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.519277 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.520051 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.521061 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.521531 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.521640 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.524388 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:44.524404 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:44.524427 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:44.524463 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:44.526637 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.527099 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.527106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.527108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.527201 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.527487 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.527491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.527493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.527758 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:44.527976 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:44.528165 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 18:53:44.528323 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 18:53:44.532310 7956 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:44.531447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.30.153.129:45931)
0810 18:53:44.531617 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.531620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.531636 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:44.531722 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.531724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.531757 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.531821 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.532204 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51198: BlockingRecv error: recv got EOF from 127.0.0.1:51198 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 18:53:44.532358 31334 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.533320 31334 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.534264 31334 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:44.542979 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.535139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.30.153.129:45931)
0810 18:53:44.535304 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.535308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.535409 (+ 101us) server_negotiation.cc:407] Connection header received
0810 18:53:44.535486 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.535488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.535518 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.535579 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.535998 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.536563 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.537321 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.537545 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.537628 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.539902 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:44.539906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:44.539908 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:44.539929 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:44.541333 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.541778 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.541787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.541788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.541860 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.542139 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.542142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.542144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.542302 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:44.542467 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:44.542726 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 18:53:44.542845 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260810 18:53:44.545639 7956 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:44.544679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.30.153.129:45931)
0810 18:53:44.544834 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.544837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.544849 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:53:44.544901 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.544902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.544964 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.545055 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.545513 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51228: BlockingRecv error: recv got EOF from 127.0.0.1:51228 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 18:53:44.545653 31334 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786387990686100-31334-0/krb5kdc/krb5cc' of type FILE
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.546648 31334 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:44.547549 31334 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388024, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:53:44.555619 7956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:44.548341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.30.153.129:45931)
0810 18:53:44.548471 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:44.548475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:44.548497 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:53:44.548608 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:44.548610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:44.548644 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:44.548709 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:44.549157 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.549644 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.550632 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:44.550838 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:44.550945 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:44.553021 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:44.553027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:44.553029 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:44.553054 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:44.554227 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.554659 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.554661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.554662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.554734 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:44.554940 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:44.554942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:44.554943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:44.555071 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:44.555164 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:44.555326 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 18:53:44.555409 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:53:44.556737 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 7392
I20260810 18:53:44.571799 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 7575
I20260810 18:53:44.579507 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 7759
I20260810 18:53:44.597685 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 7260
2026-08-10T18:53:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2484 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7975](info): set up 2 sockets
Aug 10 18:53:44 dist-test-slave-68xk krb5kdc[7975](info): commencing operation
krb5kdc: starting...
W20260810 18:53:45.860131 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:45 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.190.keytab.
Entry for principal oryx/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.190.keytab.
I20260810 18:53:45.999370 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:40429
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.190.keytab
--principal=oryx/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:40429
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:46.112296 7989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:46.112632 7989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:46.112713 7989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:46.116739 7989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:46.116847 7989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:46.116884 7989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:46.116922 7989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:46.116954 7989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:46.122357 7989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:40429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.190.keytab
--principal=oryx/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:40429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:46.123579 7989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:46.124665 7989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:46.133463 8000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:46.133530 7995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:46.133471 7997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:46.133795 7989 server_base.cc:1034] running on GCE node
I20260810 18:53:46.134564 7989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:46.135813 7989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:46.137045 7989 hybrid_clock.cc:648] HybridClock initialized: now 1786388026137021 us; error 43 us; skew 500 ppm
Aug 10 18:53:46 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:46.144090 7989 init.cc:377] Logged in from keytab as oryx/127.30.153.190@KRBTEST.COM (short username oryx)
I20260810 18:53:46.144994 7989 webserver.cc:460] Webserver started at http://127.30.153.190:38997/ using document root <none> and password file <none>
I20260810 18:53:46.145560 7989 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:46.145630 7989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:46.145807 7989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:46.147995 7989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "aa3fb5cc172547019da531d495dcef0a"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "3d6d56d62a820a29a0424152991a3ba3"
server_key_iv: "fc6a046a1a2398e7cbdcb499f3aaac14"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.148522 7989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "aa3fb5cc172547019da531d495dcef0a"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "3d6d56d62a820a29a0424152991a3ba3"
server_key_iv: "fc6a046a1a2398e7cbdcb499f3aaac14"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.152257 7989 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:53:46.154605 8009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.155807 7989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:46.155907 7989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "aa3fb5cc172547019da531d495dcef0a"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "3d6d56d62a820a29a0424152991a3ba3"
server_key_iv: "fc6a046a1a2398e7cbdcb499f3aaac14"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.155985 7989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:46.170396 7989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:46.173132 7989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:46.173282 7989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:46.181126 7989 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:40429
I20260810 18:53:46.181437 8070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:40429 every 8 connection(s)
I20260810 18:53:46.182142 7989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:46.185050 8071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:46.186703 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 7989
I20260810 18:53:46.186851 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:46.187165 31334 external_mini_cluster.cc:1428] Setting key 17477cfc00a820038a686b78b3301189
I20260810 18:53:46.192068 8071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: Bootstrap starting.
I20260810 18:53:46.194767 8071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:46.195735 8071 log.cc:824] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:53:46 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:46.197815 8071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: No bootstrap required, opened a new log
I20260810 18:53:46.202626 8071 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } }
I20260810 18:53:46.202888 8071 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:46.202955 8071 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa3fb5cc172547019da531d495dcef0a, State: Initialized, Role: FOLLOWER
I20260810 18:53:46.203361 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.188635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47716 (local address 127.30.153.190:40429)
0810 18:53:46.189081 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:46.189101 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:46.189145 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:53:46.190041 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:46.190066 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:46.190500 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:46.190881 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:46.191676 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.193110 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.194012 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.194729 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.194910 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.197700 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:46.197706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:46.197722 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:46.197753 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:46.199969 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.200513 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.200519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.200525 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.200683 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.201031 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.201035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.201036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.201550 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:46.201759 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:46.202057 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:53:46.202357 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":110,"threads_started":1}
I20260810 18:53:46.203437 8071 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } }
I20260810 18:53:46.203588 8071 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:46.203658 8071 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:46.203763 8071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:46.204783 8071 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } }
I20260810 18:53:46.205159 8071 leader_election.cc:302] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa3fb5cc172547019da531d495dcef0a; no voters:
I20260810 18:53:46.205490 8071 leader_election.cc:288] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:46.205624 8079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:46.205884 8079 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 1 LEADER]: Becoming Leader. State: Replica: aa3fb5cc172547019da531d495dcef0a, State: Running, Role: LEADER
I20260810 18:53:46.206182 8079 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } }
I20260810 18:53:46.206784 8071 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:46.208055 8080 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3fb5cc172547019da531d495dcef0a. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } }
I20260810 18:53:46.208057 8081 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } }
I20260810 18:53:46.208214 8080 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:46.208216 8081 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:46.208873 8088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:46.212126 8088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:46.217455 8088 catalog_manager.cc:1347] Generated new cluster ID: 5c7346c4c51a452b81bd5ca0dcaa61bc
I20260810 18:53:46.217518 8088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:46.228010 8088 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:46.229001 8088 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:46.235951 8088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: Generated new TSK 0
I20260810 18:53:46.236737 8088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab.
Entry for principal oryx/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab.
I20260810 18:53:46.282318 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40429
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab
--principal=oryx/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:46.404126 8104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:46.404371 8104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:46.404412 8104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:46.408524 8104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:46.408586 8104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:46.408679 8104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:46.413877 8104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab
--principal=oryx/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:46.414920 8104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:46.415970 8104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:46.425673 8115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:46.425725 8104 server_base.cc:1034] running on GCE node
W20260810 18:53:46.425681 8118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:46.425956 8116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:46.426348 8104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:46.426982 8104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:46.428170 8104 hybrid_clock.cc:648] HybridClock initialized: now 1786388026428157 us; error 30 us; skew 500 ppm
Aug 10 18:53:46 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:46.430962 8104 init.cc:377] Logged in from keytab as oryx/127.30.153.129@KRBTEST.COM (short username oryx)
I20260810 18:53:46.431800 8104 webserver.cc:460] Webserver started at http://127.30.153.129:33027/ using document root <none> and password file <none>
I20260810 18:53:46.432358 8104 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:46.432454 8104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:46.432686 8104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:46.434823 8104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "3388724472a644d38545ea985f6fb264"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "b956df136be46e8d596f2f3912895fe0"
server_key_iv: "dbe7c642a8d85f1bd34b9b021cc1f6ed"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.435638 8104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "3388724472a644d38545ea985f6fb264"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "b956df136be46e8d596f2f3912895fe0"
server_key_iv: "dbe7c642a8d85f1bd34b9b021cc1f6ed"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.439471 8104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:46.442090 8128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.443445 8104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:46.443596 8104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "3388724472a644d38545ea985f6fb264"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "b956df136be46e8d596f2f3912895fe0"
server_key_iv: "dbe7c642a8d85f1bd34b9b021cc1f6ed"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.443715 8104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:46.460126 8104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:46.463128 8104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:46.463344 8104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:46.463930 8104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:46.464761 8104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:46.464833 8104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.464903 8104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:46.464956 8104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.474484 8104 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:46261
I20260810 18:53:46.474524 8276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:46261 every 8 connection(s)
I20260810 18:53:46.475483 8104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:46.480294 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 8104
I20260810 18:53:46.480463 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:46.480768 31334 external_mini_cluster.cc:1428] Setting key 937cf53941ce44a77345051338a375ca
Aug 10 18:53:46 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:46.490895 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.477393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:54555 (local address 127.30.153.190:40429)
0810 18:53:46.477529 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:46.477533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:46.478393 (+ 860us) server_negotiation.cc:407] Connection header received
0810 18:53:46.479818 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:46.479821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:46.479881 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:46.479989 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:46.481411 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.482044 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.483350 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.483753 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.483881 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.486775 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:46.486782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:46.486788 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:46.486822 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:46.488668 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.489183 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.489189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.489192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.489296 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.489639 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.489645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.489649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.489920 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:46.490034 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:46.490557 (+ 523us) server_negotiation.cc:299] Negotiation successful
0810 18:53:46.490694 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:53:46.491623 8279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.477783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:54555)
0810 18:53:46.478237 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:46.478286 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:46.479599 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:46.480190 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:46.480197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:46.480667 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:46.481233 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:46.481263 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.482191 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.482196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:46.483202 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:46.483212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.483913 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.483916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:46.484052 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.484745 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:46.484770 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:46.486601 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:46.488786 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:46.488791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:46.488803 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:46.489057 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:46.489399 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:46.489401 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:46.489402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:46.489527 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:46.490036 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:46.490041 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:46.490278 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:53:46.490488 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 18:53:46.490743 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for oryx/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:53:46.493029 8277 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40429
I20260810 18:53:46.493280 8277 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:46.493824 8277 heartbeater.cc:511] Master 127.30.153.190:40429 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.130@KRBTEST.COM" created.
I20260810 18:53:46.495616 8028 ts_manager.cc:194] Registered new tserver with Master: 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261)
I20260810 18:53:46.496843 8028 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:54555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab.
Entry for principal oryx/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab.
I20260810 18:53:46.513765 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40429
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab
--principal=oryx/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:46.626760 8283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:46.627002 8283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:46.627044 8283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:46.631179 8283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:46.631234 8283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:46.631358 8283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:46.636518 8283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab
--principal=oryx/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:46.637514 8283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:46.638568 8283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:46.648463 8292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:46.648523 8289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:46.648626 8288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:46.648869 8283 server_base.cc:1034] running on GCE node
I20260810 18:53:46.649209 8283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:46.649874 8283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:46.651090 8283 hybrid_clock.cc:648] HybridClock initialized: now 1786388026651087 us; error 49 us; skew 500 ppm
Aug 10 18:53:46 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:46.654163 8283 init.cc:377] Logged in from keytab as oryx/127.30.153.130@KRBTEST.COM (short username oryx)
I20260810 18:53:46.655225 8283 webserver.cc:460] Webserver started at http://127.30.153.130:39833/ using document root <none> and password file <none>
I20260810 18:53:46.655784 8283 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:46.655846 8283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:46.656010 8283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:46.658098 8283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "a684fc596e4446378f25533f4a960da7"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "5888310615897ce9c935b1c704635e47"
server_key_iv: "316016d8a7e83d6056ac106801df5d30"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.658716 8283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "a684fc596e4446378f25533f4a960da7"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "5888310615897ce9c935b1c704635e47"
server_key_iv: "316016d8a7e83d6056ac106801df5d30"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.662839 8283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:53:46.665328 8303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.666617 8283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:46.666779 8283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "a684fc596e4446378f25533f4a960da7"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "5888310615897ce9c935b1c704635e47"
server_key_iv: "316016d8a7e83d6056ac106801df5d30"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.666888 8283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:46.690231 8283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:46.693336 8283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:46.693549 8283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:46.694187 8283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:46.695132 8283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:46.695181 8283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.695223 8283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:46.695240 8283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.704873 8283 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:41749
I20260810 18:53:46.704902 8461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:41749 every 8 connection(s)
I20260810 18:53:46.705866 8283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:46.710925 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 8283
I20260810 18:53:46.711066 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:46.711364 31334 external_mini_cluster.cc:1428] Setting key 72a21b2c3fa356c3e31f9bed2e49746d
Aug 10 18:53:46 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:46.720824 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.707883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:40899 (local address 127.30.153.190:40429)
0810 18:53:46.708066 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:46.708071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:46.709024 (+ 953us) server_negotiation.cc:407] Connection header received
0810 18:53:46.710498 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:46.710502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:46.710562 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:46.710644 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:46.711995 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.712710 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.713882 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.714114 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.714231 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.717146 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:46.717149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:46.717151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:46.717183 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:46.718731 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.719265 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.719270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.719271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.719357 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.719730 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.719733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.719735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.719920 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:46.720030 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:46.720536 (+ 506us) server_negotiation.cc:299] Negotiation successful
0810 18:53:46.720660 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for oryx/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:53:46.721690 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.708334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:40899)
0810 18:53:46.708872 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:46.708917 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:46.710248 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:46.710827 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:46.710835 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:46.711301 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:46.711846 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:46.711860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.712850 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.712853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:46.713745 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:46.713755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.714284 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.714288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:46.714518 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.715186 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:46.715212 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:46.716963 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:46.718855 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:46.718861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:46.718876 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:46.719156 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:46.719454 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:46.719458 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:46.719460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:46.719603 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:46.720031 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:46.720041 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:46.720292 (+ 251us) client_negotiation.cc:769] Sending connection context
0810 18:53:46.720491 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 18:53:46.720827 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":127,"threads_started":1}
I20260810 18:53:46.723021 8462 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40429
I20260810 18:53:46.723321 8462 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:46.723886 8462 heartbeater.cc:511] Master 127.30.153.190:40429 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.131@KRBTEST.COM" created.
I20260810 18:53:46.725203 8028 ts_manager.cc:194] Registered new tserver with Master: a684fc596e4446378f25533f4a960da7 (127.30.153.130:41749)
I20260810 18:53:46.725793 8028 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:40899
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab.
Entry for principal oryx/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab.
I20260810 18:53:46.742769 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40429
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab
--principal=oryx/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:46.859050 8469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:46.859359 8469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:46.859406 8469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:46.863827 8469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:46.863893 8469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:46.863950 8469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:46.869165 8469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab
--principal=oryx/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:46.870215 8469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:46.871327 8469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:46.881466 8477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:46.881537 8478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:46.881657 8469 server_base.cc:1034] running on GCE node
W20260810 18:53:46.881568 8482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:46.882191 8469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:46.882913 8469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:46.884119 8469 hybrid_clock.cc:648] HybridClock initialized: now 1786388026884107 us; error 39 us; skew 500 ppm
Aug 10 18:53:46 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:46.887075 8469 init.cc:377] Logged in from keytab as oryx/127.30.153.131@KRBTEST.COM (short username oryx)
I20260810 18:53:46.887972 8469 webserver.cc:460] Webserver started at http://127.30.153.131:36077/ using document root <none> and password file <none>
I20260810 18:53:46.888549 8469 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:46.888645 8469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:46.888888 8469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:46.891064 8469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "ceaae3ac34bf41d9857c39d16129f6c9"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "2512e2c96056896c3f4142bfdccecd06"
server_key_iv: "d4f3dab89fbccc711bd9bf519335e745"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.891628 8469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "ceaae3ac34bf41d9857c39d16129f6c9"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "2512e2c96056896c3f4142bfdccecd06"
server_key_iv: "d4f3dab89fbccc711bd9bf519335e745"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.895454 8469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:53:46.897900 8493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.899191 8469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:46.899364 8469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "ceaae3ac34bf41d9857c39d16129f6c9"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "2512e2c96056896c3f4142bfdccecd06"
server_key_iv: "d4f3dab89fbccc711bd9bf519335e745"
server_key_version: "encryptionkey@0"
I20260810 18:53:46.899477 8469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:46.910318 8469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:46.913370 8469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:46.913592 8469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:46.914202 8469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:46.915133 8469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:46.915207 8469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.915277 8469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:46.915329 8469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:46.924552 8469 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:39721
I20260810 18:53:46.924594 8651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:39721 every 8 connection(s)
I20260810 18:53:46.925483 8469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:46.930200 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 8469
I20260810 18:53:46.930346 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:46.930678 31334 external_mini_cluster.cc:1428] Setting key 0f38c8e34a7ca346156b6895f6e4e72c
Aug 10 18:53:46 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:46.939764 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.927611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:34411 (local address 127.30.153.190:40429)
0810 18:53:46.927790 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:46.927794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:46.928616 (+ 822us) server_negotiation.cc:407] Connection header received
0810 18:53:46.930055 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:46.930059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:46.930129 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:46.930233 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:46.931696 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.932231 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.933384 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.933628 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.933741 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.936313 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:46.936317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:46.936319 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:46.936351 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:46.937712 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.938228 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.938231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.938233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.938315 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:46.938694 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:46.938697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:46.938699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:46.938876 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:46.938985 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:46.939474 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 18:53:46.939592 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 18:53:46.940512 8658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:46.927955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:34411)
0810 18:53:46.928440 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:46.928482 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:46.929836 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:46.930465 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:46.930476 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:46.930962 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:46.931515 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:46.931542 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.932360 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.932364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:46.933256 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:46.933265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:46.933782 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:46.933786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:46.933958 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:46.934603 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:46.934620 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:46.936152 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:46.937831 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:46.937836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:46.937850 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:46.938114 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:46.938428 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:46.938434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:46.938436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:46.938581 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:46.938984 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:46.938992 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:46.939229 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:53:46.939415 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 18:53:46.939732 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":130,"threads_started":1}
I20260810 18:53:46.941735 8655 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40429
I20260810 18:53:46.941946 8655 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:46.942395 8655 heartbeater.cc:511] Master 127.30.153.190:40429 requested a full tablet report, sending...
I20260810 18:53:46.943529 8028 ts_manager.cc:194] Registered new tserver with Master: ceaae3ac34bf41d9857c39d16129f6c9 (127.30.153.131:39721)
I20260810 18:53:46.944154 8028 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:34411
I20260810 18:53:46.944685 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.153.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.189.keytab.
Entry for principal oryx/127.30.153.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.189.keytab.
I20260810 18:53:46.974496 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.189:39603
--webserver_interface=127.30.153.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.189.keytab
--principal=oryx/127.30.153.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:40429,127.30.153.189:39603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:47.091531 8661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:47.091823 8661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:47.091914 8661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:47.095917 8661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:47.095999 8661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:47.096038 8661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:47.096099 8661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:47.096140 8661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:47.101424 8661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:40429,127.30.153.189:39603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.189.keytab
--principal=oryx/127.30.153.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.189:39603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.153.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
W20260810 18:53:47.101678 8661 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.153.190:40429,127.30.153.189:39603'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 18:53:47.102618 8661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:47.103703 8661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:47.112820 8672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:47.112807 8667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:47.112805 8668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:47.113193 8661 server_base.cc:1034] running on GCE node
I20260810 18:53:47.113758 8661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:47.115011 8661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:47.116219 8661 hybrid_clock.cc:648] HybridClock initialized: now 1786388027116190 us; error 45 us; skew 500 ppm
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:47.119290 8661 init.cc:377] Logged in from keytab as oryx/127.30.153.189@KRBTEST.COM (short username oryx)
I20260810 18:53:47.120244 8661 webserver.cc:460] Webserver started at http://127.30.153.189:41149/ using document root <none> and password file <none>
I20260810 18:53:47.120844 8661 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:47.120944 8661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:47.121192 8661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:47.123436 8661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data/instance:
uuid: "c2ebd8ba15df4bac8d4e882ce5aac624"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "2f43c3dd12e958316e7fe8369237201d"
server_key_iv: "9025566e029a2e40743b9aa2b844cbfa"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.124033 8661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal/instance:
uuid: "c2ebd8ba15df4bac8d4e882ce5aac624"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "2f43c3dd12e958316e7fe8369237201d"
server_key_iv: "9025566e029a2e40743b9aa2b844cbfa"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.128074 8661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:47.130704 8685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.132189 8661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:53:47.132372 8661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
uuid: "c2ebd8ba15df4bac8d4e882ce5aac624"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "2f43c3dd12e958316e7fe8369237201d"
server_key_iv: "9025566e029a2e40743b9aa2b844cbfa"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.132507 8661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:47.146327 8661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:47.149744 8661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:47.149961 8661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:47.158582 8661 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.189:39603
I20260810 18:53:47.158856 8769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.189:39603 every 8 connection(s)
I20260810 18:53:47.159745 8661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
I20260810 18:53:47.161824 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 8661
I20260810 18:53:47.163365 8771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:47.167934 8771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.153.189@KRBTEST.COM
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.189@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:47.178844 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.163643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.30.153.189:39603)
0810 18:53:47.164075 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.164092 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.164129 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:53:47.164807 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.164825 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.165177 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.165456 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.166488 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.167598 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.168771 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.169274 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.169414 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.172946 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.172952 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.172961 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.172987 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.175110 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.175652 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.175658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.175662 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.175791 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.176245 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.176248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.176249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.176918 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.177213 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.177484 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.177812 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"spinlock_wait_cycles":4480,"thread_start_us":133,"threads_started":1}
I20260810 18:53:47.180953 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.169965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47730 (local address 127.30.153.190:40429)
0810 18:53:47.170086 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.170089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.170580 (+ 491us) server_negotiation.cc:407] Connection header received
0810 18:53:47.170815 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.170818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.170866 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.170937 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.171867 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.172600 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.173846 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.174185 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.174318 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.177178 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.177181 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.177183 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.177211 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.179000 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.179528 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.179532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.179534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.179626 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.179921 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.179925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.179926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.180168 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.180288 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.180626 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.180761 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 18:53:47.182595 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.169919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:47730)
0810 18:53:47.170386 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.170461 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.170702 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.171033 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.171041 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.171538 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.171710 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.171716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.172751 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.172755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.173699 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.173709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.174322 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.174324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.174476 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.175007 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.175042 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.177005 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.179155 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.179166 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.179172 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.179409 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.179726 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.179729 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.179731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.179822 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.180288 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.180296 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.180417 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.180561 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.180761 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"spinlock_wait_cycles":896,"thread_start_us":164,"threads_started":1}
I20260810 18:53:47.183971 8771 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.189@KRBTEST.COM for oryx/127.30.153.189@KRBTEST.COM
I20260810 18:53:47.193415 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.184540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.0.0.1:33328)
0810 18:53:47.184678 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.184693 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.184766 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.184968 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.184970 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.185157 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.185270 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.185275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.186471 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.186475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.187294 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.187303 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.187809 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.187813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.187909 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.188391 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.188405 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.189880 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.191803 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.191808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.191811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.192021 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.192332 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.192335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.192337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.192424 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.192824 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.192828 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.192916 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.193074 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.193227 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:53:47.193414 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.184664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33328 (local address 127.30.153.189:39603)
0810 18:53:47.184785 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.184788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.184801 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:47.184832 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.184834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.184868 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.184937 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.185385 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.186333 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.187426 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.187680 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.187785 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.190024 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.190029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.190031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.190060 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.191663 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.192138 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.192143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.192145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.192225 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.192540 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.192544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.192545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.192719 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.192835 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.193128 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.193288 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":46}
I20260810 18:53:47.196338 8771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Bootstrap starting.
I20260810 18:53:47.198753 8771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:47.199542 8771 log.cc:824] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:47.201567 8771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: No bootstrap required, opened a new log
I20260810 18:53:47.204845 8771 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } }
I20260810 18:53:47.205058 8771 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:47.205178 8771 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2ebd8ba15df4bac8d4e882ce5aac624, State: Initialized, Role: FOLLOWER
I20260810 18:53:47.205607 8771 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } }
I20260810 18:53:47.206142 8780 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } } }
I20260810 18:53:47.206287 8780 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:47.207055 8771 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:53:47.212284 8798 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:47.212384 8798 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:47.214936 8661 master_runner.cc:428] Detected that this master c2ebd8ba15df4bac8d4e882ce5aac624 is joining an existing cluster
I20260810 18:53:47.215014 8661 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.153.189:39603
I20260810 18:53:47.226739 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.217547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33344 (local address 127.30.153.189:39603)
0810 18:53:47.217687 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.217692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.218478 (+ 786us) server_negotiation.cc:407] Connection header received
0810 18:53:47.218634 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.218638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.218681 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.218773 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.219277 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.219757 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.221054 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.221390 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.221497 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.223105 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.223110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.223112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.223138 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.224872 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.225353 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.225357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.225359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.225439 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.225754 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.225757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.225759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.225970 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.226069 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.226383 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.226588 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":47}
I20260810 18:53:47.227150 8808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.217967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.0.0.1:33344)
0810 18:53:47.218351 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.218362 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.218528 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.218813 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.218816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.219035 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.219157 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.219164 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.219878 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.219881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.220922 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.220934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.221581 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.221584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.221697 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.222330 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.222351 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.222976 (+ 625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.224987 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.224990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.224992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.225248 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.225535 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.225540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.225541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.225626 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.226078 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.226081 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.226191 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.226327 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.226529 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":49,"thread_start_us":122,"threads_started":1}
I20260810 18:53:47.227265 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.217792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47746 (local address 127.30.153.190:40429)
0810 18:53:47.217925 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.217930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.218112 (+ 182us) server_negotiation.cc:407] Connection header received
0810 18:53:47.218258 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.218262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.218316 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.218444 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.219072 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.219909 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.221101 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.221323 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.221442 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.223388 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.223395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.223399 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.223430 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.224965 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.225483 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.225486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.225488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.225583 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.225893 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.225897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.225899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.226134 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.226254 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.226960 (+ 706us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.227080 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:47.227236 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.217527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:47746)
0810 18:53:47.217986 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.218015 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.218136 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.218479 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.218482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.218756 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.218939 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.218946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.220025 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.220028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.220924 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.220934 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.221463 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.221467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.221564 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.222752 (+ 1188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.222768 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.223237 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.225107 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.225113 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.225115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.225372 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.225697 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.225702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.225705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.225794 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.226258 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.226261 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.226346 (+ 85us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.226537 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.226675 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":19,"thread_start_us":120,"threads_started":1}
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.190@KRBTEST.COM for oryx/127.30.153.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.30.153.188@KRBTEST.COM; defaulting to no policy
I20260810 18:53:47.244992 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.232719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.0.0.1:33352)
0810 18:53:47.233459 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.233499 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.233685 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.234035 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.234042 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.234508 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.234644 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.234651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.235906 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.235909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.237103 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.237119 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.237738 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.237745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.237918 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.238623 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.238643 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.240876 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.243028 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.243040 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.243046 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.243413 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.243753 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.243760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.243764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.243870 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.244345 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.244353 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.244517 (+ 164us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.244685 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.244848 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":102,"threads_started":1}
I20260810 18:53:47.245047 8775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.232730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33352 (local address 127.30.153.189:39603)
0810 18:53:47.232881 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.232886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.233604 (+ 718us) server_negotiation.cc:407] Connection header received
0810 18:53:47.233834 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.233837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.233895 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.233994 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.234843 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.235727 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.237242 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.237605 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.237707 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.241038 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.241043 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.241046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.241071 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.242898 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.243546 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.243550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.243552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.243636 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.243974 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.243977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.243979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.244216 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.244316 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.244768 (+ 452us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.244885 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 18:53:47.246667 8026 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.153.189:39603: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624"
member_type: NON_VOTER
last_known_addr {
host: "127.30.153.189"
port: 39603
}
attrs {
promote: true
}
}
dest_uuid: "aa3fb5cc172547019da531d495dcef0a"
cas_config_opid_index: -1
I20260810 18:53:47.247579 8026 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.188@KRBTEST.COM" created.
I20260810 18:53:47.249809 8812 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c2ebd8ba15df4bac8d4e882ce5aac624 (127.30.153.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } }
I20260810 18:53:47.250890 8734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:47.252082 8812 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.153.189:39603
I20260810 18:53:47.252563 8817 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } }
I20260810 18:53:47.252624 8661 master.cc:537] Master@127.30.153.189:39603 shutting down...
I20260810 18:53:47.252679 8734 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Refusing update from remote peer aa3fb5cc172547019da531d495dcef0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 18:53:47.252806 8817 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:47.253432 8814 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:53:47.255383 8011 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 18:53:47.263746 8011 consensus_peers.cc:487] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a -> Peer c2ebd8ba15df4bac8d4e882ce5aac624 (127.30.153.189:39603): Couldn't send request to peer c2ebd8ba15df4bac8d4e882ce5aac624. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 18:53:47.266762 8661 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:47.267096 8661 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:47.267218 8661 tablet_replica.cc:330] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.188.keytab.
Entry for principal oryx/127.30.153.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.188.keytab.
I20260810 18:53:47.269758 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.188:35207
--webserver_interface=127.30.153.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.188.keytab
--principal=oryx/127.30.153.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:40429,127.30.153.189:39603,127.30.153.188:35207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:53:47.284868 8661 master.cc:552] Master@127.30.153.189:39603 shutdown complete.
I20260810 18:53:47.284940 8661 master_runner.cc:305] Clearing existing system tablet
I20260810 18:53:47.285250 8661 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:47.285319 8661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:47.286914 8661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260810 18:53:47.287935 8825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.288156 8661 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:47.288277 8661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:47.288312 8661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
uuid: "c2ebd8ba15df4bac8d4e882ce5aac624"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "2f43c3dd12e958316e7fe8369237201d"
server_key_iv: "9025566e029a2e40743b9aa2b844cbfa"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.290548 8661 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:47.293112 8661 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 18:53:47.293210 8661 log.cc:1196] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 18:53:47.293480 8661 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Deleting consensus metadata
I20260810 18:53:47.293663 8661 master_runner.cc:315] Copying system tablet from 127.30.153.190:40429
I20260810 18:53:47.295101 8661 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: tablet copy: Beginning tablet copy session from remote peer at address 127.30.153.190:40429
I20260810 18:53:47.305198 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.295774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:47760)
0810 18:53:47.296150 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.296166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.296297 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.296607 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.296610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.296863 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.297013 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.297021 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.298145 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.298149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.299224 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.299236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.299848 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.299853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.299964 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.300600 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.300618 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.301105 (+ 487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.303369 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.303373 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.303375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.303663 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.304003 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.304006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.304008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.304101 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.304598 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.304601 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.304701 (+ 100us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.304820 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.304965 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":99,"threads_started":1}
I20260810 18:53:47.305609 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.295914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.30.153.190:40429)
0810 18:53:47.296057 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.296062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.296263 (+ 201us) server_negotiation.cc:407] Connection header received
0810 18:53:47.296403 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.296406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.296461 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.296562 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.297172 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.297997 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.299370 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.299697 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.299823 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.301272 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.301276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.301279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.301314 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.303240 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.303787 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.303790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.303792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.303883 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.304218 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.304222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.304224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.304481 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.304599 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.304908 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.305156 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:47.306201 8052 tablet_copy_service.cc:140] P aa3fb5cc172547019da531d495dcef0a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c2ebd8ba15df4bac8d4e882ce5aac624 ({username='oryx', principal='oryx/127.30.153.189@KRBTEST.COM'} at 127.0.0.1:47760)
I20260810 18:53:47.306406 8052 tablet_copy_service.cc:161] P aa3fb5cc172547019da531d495dcef0a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c2ebd8ba15df4bac8d4e882ce5aac624 at {username='oryx', principal='oryx/127.30.153.189@KRBTEST.COM'} at 127.0.0.1:47760: session id = c2ebd8ba15df4bac8d4e882ce5aac624-00000000000000000000000000000000
I20260810 18:53:47.308634 8052 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 18:53:47.310602 8661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 18:53:47.313237 8661 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: tablet copy: Starting download of 0 data blocks...
I20260810 18:53:47.313520 8661 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: tablet copy: Starting download of 1 WAL segments...
I20260810 18:53:47.315714 8661 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 18:53:47.317745 8052 tablet_copy_service.cc:342] P aa3fb5cc172547019da531d495dcef0a: Request end of tablet copy session c2ebd8ba15df4bac8d4e882ce5aac624-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.153.189@KRBTEST.COM'} at 127.0.0.1:47760
I20260810 18:53:47.317875 8052 tablet_copy_service.cc:433] P aa3fb5cc172547019da531d495dcef0a: ending tablet copy session c2ebd8ba15df4bac8d4e882ce5aac624-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c2ebd8ba15df4bac8d4e882ce5aac624
I20260810 18:53:47.326619 8661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:47.328608 8833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:47.328723 8839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:47.329069 8661 server_base.cc:1034] running on GCE node
W20260810 18:53:47.330569 8836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:47.330850 8661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:47.331238 8661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:47.332405 8661 hybrid_clock.cc:648] HybridClock initialized: now 1786388027332392 us; error 32 us; skew 500 ppm
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:47.334204 8661 init.cc:377] Logged in from keytab as oryx/127.30.153.189@KRBTEST.COM (short username oryx)
I20260810 18:53:47.334776 8661 webserver.cc:460] Webserver started at http://127.30.153.189:44679/ using document root <none> and password file <none>
I20260810 18:53:47.334998 8661 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:47.335062 8661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:47.336700 8661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260810 18:53:47.337831 8850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.338155 8661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:47.338246 8661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
uuid: "c2ebd8ba15df4bac8d4e882ce5aac624"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "2f43c3dd12e958316e7fe8369237201d"
server_key_iv: "9025566e029a2e40743b9aa2b844cbfa"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.338341 8661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:47.368057 8661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:47.370607 8661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:47.376413 8926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.189:39603 every 8 connection(s)
I20260810 18:53:47.376607 8661 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.189:39603
I20260810 18:53:47.377341 8661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
I20260810 18:53:47.379396 8927 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:53:47.380523 8927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Bootstrap starting.
I20260810 18:53:47.392259 8927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:53:47.392585 8927 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Bootstrap complete.
I20260810 18:53:47.393167 8927 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } }
I20260810 18:53:47.393313 8927 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c2ebd8ba15df4bac8d4e882ce5aac624, State: Initialized, Role: LEARNER
I20260810 18:53:47.393416 8927 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } }
I20260810 18:53:47.393795 8930 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } }
I20260810 18:53:47.393988 8930 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:53:47.393890 8927 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:47.397490 8947 catalog_manager.cc:1259] Loaded cluster ID: 5c7346c4c51a452b81bd5ca0dcaa61bc
I20260810 18:53:47.397591 8947 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: loading cluster ID for follower catalog manager: success
I20260810 18:53:47.399771 8947 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: acquiring CA information for follower catalog manager: success
I20260810 18:53:47.401175 8947 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 18:53:47.411729 8822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:47.411976 8822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:47.412017 8822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:47.415884 8822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:47.415933 8822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:47.415952 8822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:47.415968 8822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:47.415983 8822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:47.421303 8822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:40429,127.30.153.189:39603,127.30.153.188:35207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.188.keytab
--principal=oryx/127.30.153.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.188:35207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.153.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:47.422251 8822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:47.423393 8822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:47.432226 8956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:47.432232 8959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:47.432226 8955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:47.432442 8822 server_base.cc:1034] running on GCE node
I20260810 18:53:47.433153 8822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:47.434283 8822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:47.435667 8822 hybrid_clock.cc:648] HybridClock initialized: now 1786388027435627 us; error 50 us; skew 500 ppm
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:47.438784 8822 init.cc:377] Logged in from keytab as oryx/127.30.153.188@KRBTEST.COM (short username oryx)
I20260810 18:53:47.439779 8822 webserver.cc:460] Webserver started at http://127.30.153.188:36531/ using document root <none> and password file <none>
I20260810 18:53:47.440371 8822 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:47.440434 8822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:47.440697 8822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:47.442889 8822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data/instance:
uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "b6472bc40bf75e701eda6657fa820649"
server_key_iv: "cd581b6b87dfa1397c1ce1e143b438f1"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.443496 8822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal/instance:
uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "b6472bc40bf75e701eda6657fa820649"
server_key_iv: "cd581b6b87dfa1397c1ce1e143b438f1"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.447475 8822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:53:47.449988 8968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.451272 8822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:53:47.451406 8822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "b6472bc40bf75e701eda6657fa820649"
server_key_iv: "cd581b6b87dfa1397c1ce1e143b438f1"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.451514 8822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:47.473311 8822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:47.477126 8822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:47.477345 8822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:47.485002 8822 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.188:35207
I20260810 18:53:47.485334 9036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.188:35207 every 8 connection(s)
I20260810 18:53:47.486086 8822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
I20260810 18:53:47.488924 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 8822
I20260810 18:53:47.489033 9037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:47.494340 9037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.153.188@KRBTEST.COM
I20260810 18:53:47.499445 8277 heartbeater.cc:503] Master 127.30.153.190:40429 was elected leader, sending a full tablet report...
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.188@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:47.505942 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.490652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33432 (local address 127.30.153.188:35207)
0810 18:53:47.491119 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.491133 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.491172 (+ 39us) server_negotiation.cc:407] Connection header received
0810 18:53:47.491849 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.491866 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.492196 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.492546 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.493420 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.494768 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.495725 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.496461 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.496617 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.499890 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.499900 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.499912 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.499944 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.502496 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.503000 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.503010 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.503020 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.503154 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.503601 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.503606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.503610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.504202 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.504429 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.504780 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.505088 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":154,"threads_started":1}
I20260810 18:53:47.506968 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.496241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:47766)
0810 18:53:47.496810 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.496870 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.497044 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.497412 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.497421 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.497921 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.498124 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.498133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.498940 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.498944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.499958 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.499968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.500566 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.500571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.500691 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.501306 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.501324 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.503100 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.505226 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.505235 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.505241 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.505479 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.505794 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.505798 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.505800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.505890 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.506340 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.506345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.506501 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.506646 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.506788 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":179,"threads_started":1}
I20260810 18:53:47.507004 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.496385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47766 (local address 127.30.153.190:40429)
0810 18:53:47.496542 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.496547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.496951 (+ 404us) server_negotiation.cc:407] Connection header received
0810 18:53:47.497169 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.497174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.497240 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.497335 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.498265 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.498797 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.500085 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.500410 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.500540 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.503315 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.503319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.503323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.503356 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.505091 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.505588 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.505591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.505593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.505696 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.505986 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.505990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.505991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.506234 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.506338 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.506733 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.506853 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:53:47.508174 9037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.188@KRBTEST.COM for oryx/127.30.153.189@KRBTEST.COM
I20260810 18:53:47.516713 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.508737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.0.0.1:33368)
0810 18:53:47.508873 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.508894 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.508957 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.509506 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.509509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.509689 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.509776 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.509781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.510812 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.510817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.511390 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.511398 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.511891 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.511895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.511998 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.512407 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.512418 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.513675 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.515511 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.515513 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.515515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.515697 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.515942 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.515944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.515946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.516017 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.516348 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.516350 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.516424 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.516501 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.516598 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:53:47.516801 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.508914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.30.153.189:39603)
0810 18:53:47.509260 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.509276 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.509290 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:53:47.509333 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.509336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.509382 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.509490 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.509899 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.510699 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.511522 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.511776 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.511891 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.513812 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.513815 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.513817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.513841 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.515410 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.515783 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.515785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.515787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.515861 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.516094 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.516096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.516097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.516266 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.516343 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.516534 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.516629 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":130,"threads_started":1}
I20260810 18:53:47.517544 9037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.188@KRBTEST.COM for oryx/127.30.153.188@KRBTEST.COM
I20260810 18:53:47.525523 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.518053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:35207 (local address 127.0.0.1:33446)
0810 18:53:47.518190 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.518205 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.518298 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.518550 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.518552 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.518713 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.518771 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.518775 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.519892 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.519895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.520370 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.520378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.520841 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.520844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.520958 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.521360 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.521370 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.522654 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.524267 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.524270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.524271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.524450 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.524695 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.524697 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.524699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.524779 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.525111 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.525113 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.525184 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.525265 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.525357 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":16}
I20260810 18:53:47.525533 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.518085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33446 (local address 127.30.153.188:35207)
0810 18:53:47.518193 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.518198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.518261 (+ 63us) server_negotiation.cc:407] Connection header received
0810 18:53:47.518397 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.518400 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.518452 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.518540 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.518878 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.519372 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.520490 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.520728 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.520843 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.522780 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.522783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.522785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.522806 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.524161 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.524537 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.524539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.524540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.524610 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.524860 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.524863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.524864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.525017 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.525104 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.525322 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.525412 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":41}
I20260810 18:53:47.528546 9037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Bootstrap starting.
I20260810 18:53:47.530990 9037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:47.531700 9037 log.cc:824] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:47.533532 9037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: No bootstrap required, opened a new log
I20260810 18:53:47.536518 9037 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } }
I20260810 18:53:47.536728 9037 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:47.536763 9037 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4fdc6c4bdfa4aeb9e3d98539b6d0fc9, State: Initialized, Role: FOLLOWER
I20260810 18:53:47.537184 9037 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } }
I20260810 18:53:47.537660 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } } }
I20260810 18:53:47.537801 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:47.538551 9037 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 18:53:47.543694 9061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:47.543781 9061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:47.547439 8822 master_runner.cc:428] Detected that this master c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 is joining an existing cluster
I20260810 18:53:47.547492 8822 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.153.188:35207
I20260810 18:53:47.559612 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 8104
I20260810 18:53:47.559931 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.550042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.30.153.189:39603)
0810 18:53:47.550262 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.550287 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.550630 (+ 343us) server_negotiation.cc:407] Connection header received
0810 18:53:47.550803 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.550809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.550865 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.550918 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.551768 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.552550 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554042 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.554390 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554482 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.556106 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.556111 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.556113 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.556149 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.557934 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.558359 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.558364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.558365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.558501 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.558921 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.558925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.558926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.559161 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.559271 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.559531 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.559672 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 18:53:47.559959 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.549705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.0.0.1:33370)
0810 18:53:47.550531 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.550547 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.550663 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.551083 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.551087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.551479 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.551614 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.551621 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.552693 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.552696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.553908 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.553916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.554652 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.554891 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.555442 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.555465 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.555892 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.558042 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.558047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.558049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.558257 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.558606 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.558609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.558612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.558713 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.559264 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.559267 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.559362 (+ 95us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.559467 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.559608 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":230,"thread_start_us":630,"threads_started":1}
I20260810 18:53:47.581992 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.549764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.30.153.188:35207)
0810 18:53:47.549877 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.549881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.550992 (+ 1111us) server_negotiation.cc:407] Connection header received
0810 18:53:47.551130 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.551147 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.551189 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.551243 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.551794 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.552549 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554039 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.554378 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554478 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.578131 (+ 23653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.578143 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.578146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.578172 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.580042 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.580546 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.580549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.580550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.580627 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.580950 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.580953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.580955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.581281 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.581353 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.581688 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.581818 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:53:47.582501 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.549706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47776 (local address 127.30.153.190:40429)
0810 18:53:47.549846 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.549851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.550705 (+ 854us) server_negotiation.cc:407] Connection header received
0810 18:53:47.550816 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.550819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.550874 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.550957 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.551690 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.552517 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554305 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.554705 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554804 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.578169 (+ 23365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.578173 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.578177 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.578214 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.580568 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.581086 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.581090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.581091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.581181 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.581503 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.581506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.581508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.581753 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.581827 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.582189 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.582334 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:47.582917 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:46261
--local_ip_for_outbound_sockets=127.30.153.129
--tserver_master_addrs=127.30.153.190:40429
--webserver_port=33027
--webserver_interface=127.30.153.129
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab
--principal=oryx/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603,127.30.153.188:35207 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:53:47.583019 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.549959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:47776)
0810 18:53:47.550627 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.550639 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.550732 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.551082 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.551085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.551401 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.551544 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.551552 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.552673 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.552676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.554164 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.554175 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.554979 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.555078 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.577076 (+ 21998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.577103 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.577964 (+ 861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.580719 (+ 2755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.580722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.580724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.580969 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.581300 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.581305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.581310 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.581401 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.581900 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.581902 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.581987 (+ 85us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.582664 (+ 677us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.582827 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"spinlock_wait_cycles":11776,"thread_start_us":89,"threads_started":1}
I20260810 18:53:47.583519 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.549969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:35207 (local address 127.0.0.1:33452)
0810 18:53:47.550925 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.550938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.551042 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.551376 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.551381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.551592 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.551694 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.551701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.552693 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.552696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.553905 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.553916 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.554650 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.554653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.554861 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.577067 (+ 22206us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.577088 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.577955 (+ 867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.580185 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.580188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.580190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.580441 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.580723 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.580726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.580728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.580849 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.581409 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.581412 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.581520 (+ 108us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.582532 (+ 1012us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.582918 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"mutex_wait_us":231,"spinlock_wait_cycles":6144,"thread_start_us":112,"threads_started":1}
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388026, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.190@KRBTEST.COM for oryx/127.30.153.188@KRBTEST.COM
I20260810 18:53:47.597180 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.588824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33454 (local address 127.30.153.188:35207)
0810 18:53:47.588937 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.588941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.589008 (+ 67us) server_negotiation.cc:407] Connection header received
0810 18:53:47.589203 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.589206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.589263 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.589350 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.589786 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.590286 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.591145 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.591377 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.591485 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.593593 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.593597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.593599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.593623 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.595368 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.595820 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.595825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.595827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.595912 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.596318 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.596321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.596323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.596509 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.596620 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.596899 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.597005 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:53:47.597180 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.588763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:35207 (local address 127.0.0.1:33454)
0810 18:53:47.588949 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.588965 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.589055 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.589370 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.589373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.589574 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.589669 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.589674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.590402 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.590406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.591011 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.591021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.591501 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.591505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.591608 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.592120 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.592147 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.593451 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.595487 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.595492 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.595494 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.595715 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.596042 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.596045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.596047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.596203 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.596612 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.596615 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.596731 (+ 116us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.596843 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.597006 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260810 18:53:47.599011 8026 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.153.188:35207: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9"
member_type: NON_VOTER
last_known_addr {
host: "127.30.153.188"
port: 35207
}
attrs {
promote: true
}
}
dest_uuid: "aa3fb5cc172547019da531d495dcef0a"
cas_config_opid_index: 5
I20260810 18:53:47.599648 8026 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } }
I20260810 18:53:47.600796 8812 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.602085 9002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:47.602403 8812 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.153.188:35207
I20260810 18:53:47.602645 8812 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.602741 8812 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:47.603703 9002 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 FOLLOWER]: Refusing update from remote peer aa3fb5cc172547019da531d495dcef0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 18:53:47.604406 8817 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:47.609292 9002 raft_consensus.cc:858] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } }
W20260810 18:53:47.609403 9002 raft_consensus.cc:1565] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } }
I20260810 18:53:47.611125 9002 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c2ebd8ba15df4bac8d4e882ce5aac624 (127.30.153.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } }
I20260810 18:53:47.611760 9077 mvcc.cc:205] Tried to move back new op lower bound from 7317045355458097152 to 7317045355341590528. Current Snapshot: MvccSnapshot[applied={T|T < 7317045355377094656}]
I20260810 18:53:47.613476 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3fb5cc172547019da531d495dcef0a. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } }
I20260810 18:53:47.613598 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 18:53:47.613706 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } }
I20260810 18:53:47.613785 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 18:53:47.613867 9045 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } }
I20260810 18:53:47.613936 9045 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 18:53:47.615589 8822 master.cc:537] Master@127.30.153.188:35207 shutting down...
I20260810 18:53:47.616727 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.601244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33384 (local address 127.30.153.189:39603)
0810 18:53:47.601368 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.601373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.604327 (+ 2954us) server_negotiation.cc:407] Connection header received
0810 18:53:47.604551 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.604555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.604631 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.604726 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:47.605176 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.605878 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.615346 (+ 9468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.616085 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.616237 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.616389 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.616520 (+ 131us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.616599 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:47.616727 8809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.600989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.0.0.1:33384)
0810 18:53:47.604248 (+ 3259us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.604263 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.604407 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.604904 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.604906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.604925 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:47.605015 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.605021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.614137 (+ 9116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.614141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.615175 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.615191 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.616249 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.616254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.616350 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.616414 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.616503 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.616590 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3167}
I20260810 18:53:47.617502 8878 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 LEARNER]: Refusing update from remote peer aa3fb5cc172547019da531d495dcef0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 18:53:47.618039 8817 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 18:53:47.619997 8878 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.622201 8930 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3fb5cc172547019da531d495dcef0a. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.622318 8817 raft_consensus.cc:1050] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: attempting to promote NON_VOTER c2ebd8ba15df4bac8d4e882ce5aac624 to VOTER
I20260810 18:53:47.622327 8930 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:53:47.622455 8930 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: NON_VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: true } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.622537 8930 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:53:47.622804 8817 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } }
W20260810 18:53:47.624794 8013 consensus_peers.cc:487] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a -> Peer c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188:35207): Couldn't send request to peer c4fdc6c4bdfa4aeb9e3d98539b6d0fc9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 18:53:47.624919 8878 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 LEARNER]: Refusing update from remote peer aa3fb5cc172547019da531d495dcef0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 18:53:47.625349 8814 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 18:53:47.627168 8814 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c2ebd8ba15df4bac8d4e882ce5aac624 (127.30.153.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.628777 8814 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.628880 8814 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:47.629000 8814 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.629076 8814 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:47.630031 8822 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260810 18:53:47.630321 8822 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260810 18:53:47.630462 8822 tablet_replica.cc:330] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: stopping tablet replica
I20260810 18:53:47.630225 8893 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c2ebd8ba15df4bac8d4e882ce5aac624 (127.30.153.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.634531 9088 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.634686 9088 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:47.646912 8822 master.cc:552] Master@127.30.153.188:35207 shutdown complete.
I20260810 18:53:47.646999 8822 master_runner.cc:305] Clearing existing system tablet
I20260810 18:53:47.647302 8822 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:47.647390 8822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:47.648824 8822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:47.649771 9091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.650008 8822 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:47.650139 8822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:47.650175 8822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "b6472bc40bf75e701eda6657fa820649"
server_key_iv: "cd581b6b87dfa1397c1ce1e143b438f1"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.651741 8822 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:47.653716 8822 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 18:53:47.653823 8822 log.cc:1196] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 18:53:47.654093 8822 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Deleting consensus metadata
I20260810 18:53:47.654306 8822 master_runner.cc:315] Copying system tablet from 127.30.153.190:40429
I20260810 18:53:47.655457 8822 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: tablet copy: Beginning tablet copy session from remote peer at address 127.30.153.190:40429
I20260810 18:53:47.663985 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.656036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:47792)
0810 18:53:47.656352 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.656366 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.656511 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.656790 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.656793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.657014 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.657101 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.657107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.657895 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.657900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.658592 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.658602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.659161 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.659165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.659246 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.659856 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.659868 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.660489 (+ 621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.662364 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.662367 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.662370 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.662636 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.662938 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.662942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.662944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.663034 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.663429 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.663432 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.663539 (+ 107us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.663650 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.663804 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":91,"threads_started":1}
I20260810 18:53:47.663983 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.656128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.30.153.190:40429)
0810 18:53:47.656293 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.656299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.656459 (+ 160us) server_negotiation.cc:407] Connection header received
0810 18:53:47.656618 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.656621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.656677 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.656769 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.657233 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.657780 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.658714 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.659016 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.659153 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.660641 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.660644 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.660647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.660675 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.662233 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.662739 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.662750 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.662752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.662840 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.663128 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.663132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.663134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.663329 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.663438 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.663706 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.663830 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 18:53:47.664880 8052 tablet_copy_service.cc:140] P aa3fb5cc172547019da531d495dcef0a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 ({username='oryx', principal='oryx/127.30.153.188@KRBTEST.COM'} at 127.0.0.1:47792)
I20260810 18:53:47.665000 8052 tablet_copy_service.cc:161] P aa3fb5cc172547019da531d495dcef0a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 at {username='oryx', principal='oryx/127.30.153.188@KRBTEST.COM'} at 127.0.0.1:47792: session id = c4fdc6c4bdfa4aeb9e3d98539b6d0fc9-00000000000000000000000000000000
I20260810 18:53:47.665886 8052 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 18:53:47.667117 8822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 18:53:47.669081 8822 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: tablet copy: Starting download of 0 data blocks...
I20260810 18:53:47.669304 8822 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: tablet copy: Starting download of 1 WAL segments...
I20260810 18:53:47.670895 8822 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 18:53:47.672644 8052 tablet_copy_service.cc:342] P aa3fb5cc172547019da531d495dcef0a: Request end of tablet copy session c4fdc6c4bdfa4aeb9e3d98539b6d0fc9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.153.188@KRBTEST.COM'} at 127.0.0.1:47792
I20260810 18:53:47.672752 8052 tablet_copy_service.cc:433] P aa3fb5cc172547019da531d495dcef0a: ending tablet copy session c4fdc6c4bdfa4aeb9e3d98539b6d0fc9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c4fdc6c4bdfa4aeb9e3d98539b6d0fc9
I20260810 18:53:47.680027 8822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:47.681661 9106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:47.681867 9102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:47.682219 8822 server_base.cc:1034] running on GCE node
W20260810 18:53:47.682183 9103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:47.682582 8822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:47.682912 8822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:47.684054 8822 hybrid_clock.cc:648] HybridClock initialized: now 1786388027684066 us; error 47 us; skew 500 ppm
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:47.685539 8822 init.cc:377] Logged in from keytab as oryx/127.30.153.188@KRBTEST.COM (short username oryx)
I20260810 18:53:47.686004 8822 webserver.cc:460] Webserver started at http://127.30.153.188:44011/ using document root <none> and password file <none>
I20260810 18:53:47.686225 8822 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:47.686275 8822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:47.687476 8822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:47.688443 9115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.688673 8822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:47.688773 8822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9"
format_stamp: "Formatted at 2026-08-10 18:53:47 on dist-test-slave-68xk"
server_key: "b6472bc40bf75e701eda6657fa820649"
server_key_iv: "cd581b6b87dfa1397c1ce1e143b438f1"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.688879 8822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:47.706501 8822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:47.709080 8822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:47.714363 8822 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.188:35207
I20260810 18:53:47.714399 9186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.188:35207 every 8 connection(s)
I20260810 18:53:47.714968 8822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
I20260810 18:53:47.716495 9187 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:53:47.717366 9187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Bootstrap starting.
I20260810 18:53:47.723879 9187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:53:47.724166 9187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: Bootstrap complete.
I20260810 18:53:47.724650 9187 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } }
I20260810 18:53:47.724781 9187 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c4fdc6c4bdfa4aeb9e3d98539b6d0fc9, State: Initialized, Role: LEARNER
I20260810 18:53:47.724902 9187 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } }
I20260810 18:53:47.725270 9191 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:47.725440 9187 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:47.725445 9191 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: LEARNER
I20260810 18:53:47.728145 8462 heartbeater.cc:503] Master 127.30.153.190:40429 was elected leader, sending a full tablet report...
I20260810 18:53:47.730321 9213 catalog_manager.cc:1259] Loaded cluster ID: 5c7346c4c51a452b81bd5ca0dcaa61bc
I20260810 18:53:47.730397 9213 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: loading cluster ID for follower catalog manager: success
I20260810 18:53:47.732227 9213 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: acquiring CA information for follower catalog manager: success
I20260810 18:53:47.733342 9213 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 18:53:47.749881 9076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:47.750226 9076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:47.750288 9076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:47.756389 9076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:47.756475 9076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:47.756560 9076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:47.763514 9076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab
--principal=oryx/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:46261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=33027
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603,127.30.153.188:35207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:47.764546 9076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:47.765549 9076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:47.775144 9224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:47.775213 9220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:47.775199 9222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:47.775344 9076 server_base.cc:1034] running on GCE node
I20260810 18:53:47.775861 9076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:47.776486 9076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:47.777637 9076 hybrid_clock.cc:648] HybridClock initialized: now 1786388027777621 us; error 26 us; skew 500 ppm
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:47.780514 9076 init.cc:377] Logged in from keytab as oryx/127.30.153.129@KRBTEST.COM (short username oryx)
I20260810 18:53:47.781392 9076 webserver.cc:460] Webserver started at http://127.30.153.129:33027/ using document root <none> and password file <none>
I20260810 18:53:47.782004 9076 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:47.782084 9076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:47.785987 9076 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:53:47.788074 9232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.789465 9076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:47.789572 9076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "3388724472a644d38545ea985f6fb264"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "b956df136be46e8d596f2f3912895fe0"
server_key_iv: "dbe7c642a8d85f1bd34b9b021cc1f6ed"
server_key_version: "encryptionkey@0"
I20260810 18:53:47.790081 9076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:47.828131 9076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:47.831614 9076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:47.831782 9076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:47.832446 9076 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:47.833338 9076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:47.833388 9076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.833428 9076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:47.833443 9076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:47.843439 9076 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:46261
I20260810 18:53:47.843489 9392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:46261 every 8 connection(s)
I20260810 18:53:47.844532 9076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:47.852844 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 9076
I20260810 18:53:47.853051 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 8283
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:47.871121 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.846892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:54991 (local address 127.30.153.190:40429)
0810 18:53:47.847049 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.847054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.847803 (+ 749us) server_negotiation.cc:407] Connection header received
0810 18:53:47.849122 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.849126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.849191 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.849280 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.851271 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.852204 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.861268 (+ 9064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.861534 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.861605 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.865472 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.865478 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.865484 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.865516 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.867994 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.868719 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.868725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.868730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.868836 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.869388 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.869395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.869401 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.869773 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.869920 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.870597 (+ 677us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.870812 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:53:47.871834 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.847019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:54991)
0810 18:53:47.847633 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.847680 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.848859 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.849453 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.849460 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.850213 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.851083 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.851099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.852332 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.852335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.861075 (+ 8740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.861090 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.861829 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.861832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.862040 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.863334 (+ 1294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.863355 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.865264 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.868116 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.868124 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.868143 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.868555 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.868954 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.868961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.868966 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.869249 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.869902 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.869911 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.870236 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.870446 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.870781 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":88,"spinlock_wait_cycles":60160,"thread_start_us":724,"threads_started":1}
I20260810 18:53:47.876094 9394 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40429
I20260810 18:53:47.876430 9394 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:47.877044 9394 heartbeater.cc:511] Master 127.30.153.190:40429 requested a full tablet report, sending...
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.189@KRBTEST.COM
I20260810 18:53:47.878763 8026 ts_manager.cc:194] Re-registered known tserver with Master: 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261)
I20260810 18:53:47.879798 8026 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:54991
Aug 10 18:53:47 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388027, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.188@KRBTEST.COM
I20260810 18:53:47.884227 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.847282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:44589 (local address 127.30.153.189:39603)
0810 18:53:47.847432 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.847437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.848518 (+ 1081us) server_negotiation.cc:407] Connection header received
0810 18:53:47.849635 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.849637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.849696 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.849790 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.851255 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.851808 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.866795 (+ 14987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.867208 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.867303 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.878785 (+ 11482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.878789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.878792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.878828 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.881515 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.882735 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.882743 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.882748 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.882862 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.883220 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.883227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.883232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.883532 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.883666 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.883961 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.884084 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:53:47.884246 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.847019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.30.153.129:44589)
0810 18:53:47.848392 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.848407 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.848881 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.849816 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.849819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.850221 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.851080 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.851096 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.851962 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.851968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.866599 (+ 14631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.866609 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.867496 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.867502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.867609 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.875394 (+ 7785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.875415 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.878600 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.881975 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.881979 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.881982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.882587 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.882987 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.882993 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.882996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.883093 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.883652 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.883656 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.883761 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.883881 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.884038 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1280,"mutex_wait_us":1233,"spinlock_wait_cycles":20352,"thread_start_us":91,"threads_started":1}
I20260810 18:53:47.885211 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.847205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:35207 (local address 127.30.153.129:50719)
0810 18:53:47.848023 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:47.848038 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:47.848859 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:47.849590 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:47.849596 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:47.850213 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:47.851083 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.851106 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.852556 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.852562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.853498 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:47.853512 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.877445 (+ 23933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.877452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:47.877580 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.878159 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:47.878183 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:47.881017 (+ 2834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:47.883226 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.883229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.883231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.883431 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.883770 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:47.883776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:47.883781 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:47.883907 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:47.884508 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:47.884515 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:47.884634 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 18:53:47.884742 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 18:53:47.884922 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":751,"mutex_wait_us":184,"spinlock_wait_cycles":9216,"thread_start_us":127,"threads_started":1}
I20260810 18:53:47.885318 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:41749
--local_ip_for_outbound_sockets=127.30.153.130
--tserver_master_addrs=127.30.153.190:40429
--webserver_port=39833
--webserver_interface=127.30.153.130
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab
--principal=oryx/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603,127.30.153.188:35207 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:53:47.885986 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:47.846637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:50719 (local address 127.30.153.188:35207)
0810 18:53:47.847132 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:47.847156 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:47.848130 (+ 974us) server_negotiation.cc:407] Connection header received
0810 18:53:47.849406 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:47.849409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:47.849455 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:47.849525 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:47.851270 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.852438 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.876833 (+ 24395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:47.877231 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:47.877367 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:47.881219 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:47.881224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:47.881227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:47.881262 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:47.883103 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.883543 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.883549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.883556 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.883674 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:47.884047 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:47.884054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:47.884060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:47.884348 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:47.884453 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:47.885525 (+ 1072us) server_negotiation.cc:299] Negotiation successful
0810 18:53:47.885691 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":98,"threads_started":1}
I20260810 18:53:47.886843 9393 heartbeater.cc:347] Connected to a master server at 127.30.153.188:35207
I20260810 18:53:47.886967 9393 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:47.887286 9393 heartbeater.cc:511] Master 127.30.153.188:35207 requested a full tablet report, sending...
I20260810 18:53:47.887290 9397 heartbeater.cc:347] Connected to a master server at 127.30.153.189:39603
I20260810 18:53:47.887470 9397 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:47.887676 9397 heartbeater.cc:511] Master 127.30.153.189:39603 requested a full tablet report, sending...
I20260810 18:53:47.888761 9131 ts_manager.cc:194] Registered new tserver with Master: 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261)
I20260810 18:53:47.888814 8865 ts_manager.cc:194] Registered new tserver with Master: 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261)
I20260810 18:53:47.946573 8655 heartbeater.cc:503] Master 127.30.153.190:40429 was elected leader, sending a full tablet report...
W20260810 18:53:48.002748 9404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:48.003034 9404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:48.003108 9404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:48.009096 9404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:48.009181 9404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:48.009294 9404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:48.019292 9404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab
--principal=oryx/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=39833
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603,127.30.153.188:35207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:48.020885 9404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:48.022395 9404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:48.035386 9412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:48.035408 9411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:48.037331 9416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:48.038905 9404 server_base.cc:1034] running on GCE node
I20260810 18:53:48.039378 9404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:48.039996 9404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:48.041181 9404 hybrid_clock.cc:648] HybridClock initialized: now 1786388028041143 us; error 51 us; skew 500 ppm
Aug 10 18:53:48 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:48.045192 9404 init.cc:377] Logged in from keytab as oryx/127.30.153.130@KRBTEST.COM (short username oryx)
I20260810 18:53:48.046175 9404 webserver.cc:460] Webserver started at http://127.30.153.130:39833/ using document root <none> and password file <none>
I20260810 18:53:48.046895 9404 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:48.047009 9404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:48.051380 9404 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:53:48.053763 9430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:48.055285 9404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:48.055440 9404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "a684fc596e4446378f25533f4a960da7"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "5888310615897ce9c935b1c704635e47"
server_key_iv: "316016d8a7e83d6056ac106801df5d30"
server_key_version: "encryptionkey@0"
I20260810 18:53:48.056010 9404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:48.078594 9404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:48.082115 9404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:48.082342 9404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:48.083055 9404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:48.084051 9404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:48.084136 9404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:48.084187 9404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:48.084204 9404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:48.095175 9404 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:41749
I20260810 18:53:48.095197 9592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:41749 every 8 connection(s)
I20260810 18:53:48.096439 9404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:48.104626 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 9404
I20260810 18:53:48.104799 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 8469
Aug 10 18:53:48 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.189@KRBTEST.COM
Aug 10 18:53:48 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.188@KRBTEST.COM
I20260810 18:53:48.114750 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.100030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:36953 (local address 127.30.153.189:39603)
0810 18:53:48.100185 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.100190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.100321 (+ 131us) server_negotiation.cc:407] Connection header received
0810 18:53:48.101145 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.101149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.101207 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.101310 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.102855 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.103725 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.105267 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.105750 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.105859 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.110140 (+ 4281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.110146 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.110151 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.110184 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.112259 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.112882 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.112888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.112892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.112974 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.113347 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.113352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.113356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.113629 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.113704 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.114359 (+ 655us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.114517 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:53:48.115131 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.098816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:39259 (local address 127.30.153.188:35207)
0810 18:53:48.098951 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.098957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.099631 (+ 674us) server_negotiation.cc:407] Connection header received
0810 18:53:48.101175 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.101179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.101239 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.101291 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.102900 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.103726 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.105504 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.105952 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.106041 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.110285 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.110294 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.110301 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.110344 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.112565 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.113253 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.113260 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.113265 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.113395 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.113827 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.113834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.113838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.114177 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.114291 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.114790 (+ 499us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.114945 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:53:48.116446 9599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.099521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.30.153.130:36953)
0810 18:53:48.100231 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.100244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.100875 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.101502 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.101509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.102002 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:48.102708 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.102726 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.104038 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.104041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.105116 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.105132 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.106135 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.106143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.106510 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.107279 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:48.107297 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:48.109951 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:48.112391 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.112399 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.112418 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.112774 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.113077 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.113082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.113086 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.113252 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.113787 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:48.113796 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:48.114105 (+ 309us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.115005 (+ 900us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.115601 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":435,"thread_start_us":94,"threads_started":1}
I20260810 18:53:48.116448 9598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.098907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:35207 (local address 127.30.153.130:39259)
0810 18:53:48.099471 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.099526 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.100858 (+ 1332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.101503 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.101510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.102009 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:48.102718 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.102726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.103822 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.103825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.105357 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.105371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.106206 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.106211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.106502 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.107180 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:48.107210 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:48.109980 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:48.112732 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.112739 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.112744 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.113096 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.113548 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.113554 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.113559 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.113679 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.114372 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:48.114377 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:48.114549 (+ 172us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.114687 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.115490 (+ 803us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":791,"spinlock_wait_cycles":13568,"thread_start_us":138,"threads_started":1}
I20260810 18:53:48.117813 9595 heartbeater.cc:347] Connected to a master server at 127.30.153.189:39603
I20260810 18:53:48.117863 9593 heartbeater.cc:347] Connected to a master server at 127.30.153.188:35207
I20260810 18:53:48.118067 9595 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:48.118072 9593 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:48.118779 9593 heartbeater.cc:511] Master 127.30.153.188:35207 requested a full tablet report, sending...
I20260810 18:53:48.118779 9595 heartbeater.cc:511] Master 127.30.153.189:39603 requested a full tablet report, sending...
I20260810 18:53:48.120173 9131 ts_manager.cc:194] Registered new tserver with Master: a684fc596e4446378f25533f4a960da7 (127.30.153.130:41749)
I20260810 18:53:48.120318 8865 ts_manager.cc:194] Registered new tserver with Master: a684fc596e4446378f25533f4a960da7 (127.30.153.130:41749)
I20260810 18:53:48.125473 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:39721
--local_ip_for_outbound_sockets=127.30.153.131
--tserver_master_addrs=127.30.153.190:40429
--webserver_port=36077
--webserver_interface=127.30.153.131
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab
--principal=oryx/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603,127.30.153.188:35207 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:53:48 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:48.131596 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.098394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:35635 (local address 127.30.153.190:40429)
0810 18:53:48.098585 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.098591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.100095 (+ 1504us) server_negotiation.cc:407] Connection header received
0810 18:53:48.101115 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.101119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.101180 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.101238 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.102887 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.103498 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.123543 (+ 20045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.123811 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.123916 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.127643 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.127646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.127648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.127675 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.129552 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.130094 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.130098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.130147 (+ 49us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.130237 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.130657 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.130660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.130661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.130865 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.130990 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.131304 (+ 314us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.131435 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 18:53:48.131942 9600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.098907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:35635)
0810 18:53:48.099979 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.099994 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.100858 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.101503 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.101510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.102009 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:48.102717 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.102726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.103664 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.103692 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.123370 (+ 19678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.123387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.123978 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.123985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.124116 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.124792 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:48.124813 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:48.127468 (+ 2655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:48.129698 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.129705 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.129711 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.129958 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.130355 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.130359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.130361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.130527 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.130995 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:48.131000 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:48.131112 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.131227 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.131409 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":229,"spinlock_wait_cycles":5248,"thread_start_us":199,"threads_started":1}
I20260810 18:53:48.132845 9594 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40429
I20260810 18:53:48.132965 9594 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:48.133225 9594 heartbeater.cc:511] Master 127.30.153.190:40429 requested a full tablet report, sending...
I20260810 18:53:48.134037 8026 ts_manager.cc:194] Re-registered known tserver with Master: a684fc596e4446378f25533f4a960da7 (127.30.153.130:41749)
I20260810 18:53:48.134752 8026 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:35635
W20260810 18:53:48.246786 9602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:48.247084 9602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:48.247157 9602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:53:48.251399 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.247178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:35207 (local address 127.0.0.1:33460)
0810 18:53:48.247415 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.247438 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.247558 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.247917 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.247920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.247944 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:48.248044 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.248049 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.249174 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.249177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.249994 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.250004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.250974 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.250977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.251053 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.251077 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.251179 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.251251 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260810 18:53:48.251416 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.247257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.30.153.188:35207)
0810 18:53:48.247388 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.247392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.247536 (+ 144us) server_negotiation.cc:407] Connection header received
0810 18:53:48.247669 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.247673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.247741 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.247859 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:48.248218 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.249009 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.250139 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.250830 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.250953 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.251105 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.251217 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.251283 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:48.252146 9151 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
W20260810 18:53:48.252655 9602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:48.252738 9602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260810 18:53:48.252625 9610 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa3fb5cc172547019da531d495dcef0a. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:48.252810 9610 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: LEARNER
W20260810 18:53:48.252831 9602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:48.258247 9602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab
--principal=oryx/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:39721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=36077
--tserver_master_addrs=127.30.153.190:40429,127.30.153.189:39603,127.30.153.188:35207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:48.259438 9602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:48.260675 9602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:48.271168 9612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:48.271245 9613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:48.271215 9617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:48.271327 9602 server_base.cc:1034] running on GCE node
I20260810 18:53:48.271812 9602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:48.272450 9602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:48.273658 9602 hybrid_clock.cc:648] HybridClock initialized: now 1786388028273630 us; error 42 us; skew 500 ppm
Aug 10 18:53:48 dist-test-slave-68xk krb5kdc[7975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:48.276559 9602 init.cc:377] Logged in from keytab as oryx/127.30.153.131@KRBTEST.COM (short username oryx)
I20260810 18:53:48.277434 9602 webserver.cc:460] Webserver started at http://127.30.153.131:36077/ using document root <none> and password file <none>
I20260810 18:53:48.278012 9602 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:48.278112 9602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:48.282104 9602 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:48.284348 9625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:48.285650 9602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:48.285802 9602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "ceaae3ac34bf41d9857c39d16129f6c9"
format_stamp: "Formatted at 2026-08-10 18:53:46 on dist-test-slave-68xk"
server_key: "2512e2c96056896c3f4142bfdccecd06"
server_key_iv: "d4f3dab89fbccc711bd9bf519335e745"
server_key_version: "encryptionkey@0"
I20260810 18:53:48.286355 9602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:48.316110 9602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:48.319582 9602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:48.319809 9602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:48.320458 9602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:48.321389 9602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:48.321463 9602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:48.321535 9602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:48.321587 9602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:48.331637 9602 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:39721
I20260810 18:53:48.331693 9790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:39721 every 8 connection(s)
I20260810 18:53:48.332739 9602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:48.333688 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 9602
Aug 10 18:53:48 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.189@KRBTEST.COM
Aug 10 18:53:48 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
Aug 10 18:53:48 dist-test-slave-68xk krb5kdc[7975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388028, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.188@KRBTEST.COM
I20260810 18:53:48.351722 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.336356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33400 (local address 127.30.153.189:39603)
0810 18:53:48.337664 (+ 1308us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.337669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.338047 (+ 378us) server_negotiation.cc:407] Connection header received
0810 18:53:48.338085 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.338088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.338126 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.338204 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.339780 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.340792 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.341989 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.342352 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.342474 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.345505 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.345509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.345512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.345540 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.347968 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.349600 (+ 1632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.349606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.349610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.349782 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.350147 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.350152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.350155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.350489 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.350644 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.351306 (+ 662us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.351444 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":1233,"thread_start_us":79,"threads_started":1}
I20260810 18:53:48.353695 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.336605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.30.153.188:35207)
0810 18:53:48.337017 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.337021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.337064 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:53:48.337114 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.337118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.337177 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.339230 (+ 2053us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.341267 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.342022 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.345801 (+ 3779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.346223 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.346347 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.348289 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.348296 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.348299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.348328 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.350909 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.351947 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.351950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.351952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.352030 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.352333 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.352336 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.352338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.352548 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.352674 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.353363 (+ 689us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.353489 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":77,"threads_started":1}
I20260810 18:53:48.353965 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.335116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:52199 (local address 127.30.153.190:40429)
0810 18:53:48.335420 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.335427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.336787 (+ 1360us) server_negotiation.cc:407] Connection header received
0810 18:53:48.337606 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.337612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.337667 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.337716 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.339875 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.340716 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.343203 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.343548 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.343647 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.347943 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.347949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.347951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.347977 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.351217 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.351926 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.351931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.351932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.352019 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.352649 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.352652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.352654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.352865 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.352952 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.353689 (+ 737us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.353822 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212}
I20260810 18:53:48.355274 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.335265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:50453 (local address 127.30.153.188:35207)
0810 18:53:48.335423 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.335428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.337219 (+ 1791us) server_negotiation.cc:407] Connection header received
0810 18:53:48.337528 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.337532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.337590 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.337643 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.342319 (+ 4676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.343407 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.348219 (+ 4812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.348592 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.348681 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.351295 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.351299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.351301 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.351325 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.353205 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.353735 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.353741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.353745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.353838 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.354303 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.354307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.354308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.354555 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.354653 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.354998 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.355115 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:53:48.355834 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.337339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.30.153.190:40429)
0810 18:53:48.338364 (+ 1025us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.338369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.339713 (+ 1344us) server_negotiation.cc:407] Connection header received
0810 18:53:48.339763 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.339767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.339827 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.339916 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.340803 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.341545 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.342676 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.343033 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.343153 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.345763 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.345767 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.345769 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.345796 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.351601 (+ 5805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.353381 (+ 1780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.353388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.353391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.353478 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.354546 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.354549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.354551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.354779 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.354926 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.355525 (+ 599us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.355653 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":941,"thread_start_us":67,"threads_started":1}
I20260810 18:53:48.355872 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.336784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:35207 (local address 127.30.153.131:50453)
0810 18:53:48.337138 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.337151 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.337300 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.338062 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.338071 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.339568 (+ 1497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:48.342193 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.342203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.343804 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.343808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.348063 (+ 4255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.348089 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.348885 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.348890 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.348990 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.349462 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:48.349477 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:48.351132 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:48.353326 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.353332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.353337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.353617 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.353946 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.353949 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.353950 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.354191 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.354715 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:48.354718 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:48.354802 (+ 84us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.355470 (+ 668us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.355635 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":5610,"thread_start_us":103,"threads_started":1}
I20260810 18:53:48.356033 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.335618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:52199)
0810 18:53:48.336693 (+ 1075us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.336709 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.337473 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.338062 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.338071 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.338877 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:48.339720 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.339730 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.340860 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.340863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.343027 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.343036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.343797 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.343800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.344133 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.344913 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:48.344924 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:48.347788 (+ 2864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:48.351419 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.351426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.351444 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.351828 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.352340 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.352344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.352346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.352524 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.353022 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:48.353029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:48.353390 (+ 361us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.354127 (+ 737us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.354576 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":1365,"spinlock_wait_cycles":25984,"thread_start_us":153,"threads_started":1}
I20260810 18:53:48.357112 9793 heartbeater.cc:347] Connected to a master server at 127.30.153.188:35207
I20260810 18:53:48.357331 9795 heartbeater.cc:347] Connected to a master server at 127.30.153.190:40429
I20260810 18:53:48.357450 9795 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:48.357265 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.335121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:49871 (local address 127.30.153.189:39603)
0810 18:53:48.335528 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.335533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.336223 (+ 690us) server_negotiation.cc:407] Connection header received
0810 18:53:48.337541 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.337545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.337600 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.337655 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.339863 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.340671 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.343243 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.343570 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.343642 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.348520 (+ 4878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.348524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.348528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.348557 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.350449 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.355190 (+ 4741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.355194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.355196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.355276 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.355799 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.355802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.355804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.356013 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.356130 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.356976 (+ 846us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.357115 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":328}
I20260810 18:53:48.357998 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.335377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.30.153.131:49871)
0810 18:53:48.336066 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.336129 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.337306 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.338063 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.338071 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.338865 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:48.339706 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.339729 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.340803 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.340809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.342233 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.342243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.343797 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.343800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.344131 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.344842 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:48.344887 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:48.348222 (+ 3335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:48.354301 (+ 6079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.354304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.354306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.355055 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.355585 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:48.355591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:48.355594 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:48.355680 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:48.356693 (+ 1013us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:48.356696 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:48.356786 (+ 90us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.357622 (+ 836us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.357778 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":221,"spinlock_wait_cycles":6016,"thread_start_us":712,"threads_started":1}
I20260810 18:53:48.358057 9795 heartbeater.cc:511] Master 127.30.153.190:40429 requested a full tablet report, sending...
I20260810 18:53:48.357367 9793 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:48.358624 9793 heartbeater.cc:511] Master 127.30.153.188:35207 requested a full tablet report, sending...
I20260810 18:53:48.358839 9796 heartbeater.cc:347] Connected to a master server at 127.30.153.189:39603
I20260810 18:53:48.358911 9796 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:48.359109 9796 heartbeater.cc:511] Master 127.30.153.189:39603 requested a full tablet report, sending...
I20260810 18:53:48.359093 8028 ts_manager.cc:194] Re-registered known tserver with Master: ceaae3ac34bf41d9857c39d16129f6c9 (127.30.153.131:39721)
I20260810 18:53:48.359404 9131 ts_manager.cc:194] Registered new tserver with Master: ceaae3ac34bf41d9857c39d16129f6c9 (127.30.153.131:39721)
I20260810 18:53:48.359845 8865 ts_manager.cc:194] Registered new tserver with Master: ceaae3ac34bf41d9857c39d16129f6c9 (127.30.153.131:39721)
I20260810 18:53:48.359836 8028 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:52199
I20260810 18:53:48.359108 8026 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:48.363288 8026 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:48.365963 9603 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: NON_VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: true } } }
I20260810 18:53:48.366122 9603 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:48.367916 9819 raft_consensus.cc:1050] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: attempting to promote NON_VOTER c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 to VOTER
I20260810 18:53:48.368315 9819 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } }
I20260810 18:53:48.369482 9151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 LEARNER]: Refusing update from remote peer aa3fb5cc172547019da531d495dcef0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 18:53:48.369493 8893 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Refusing update from remote peer aa3fb5cc172547019da531d495dcef0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260810 18:53:48.369874 9819 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 18:53:48.369993 9816 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 18:53:48.371889 9819 raft_consensus.cc:2932] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:48.373240 9151 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:48.373872 8893 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:48.374595 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:48.374747 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:48.374941 9610 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:48.375061 9610 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:48.375551 9830 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:48.375792 9830 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:48.376988 9819 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:48.377121 9819 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:48.377223 9819 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa3fb5cc172547019da531d495dcef0a" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:48.377296 9819 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:48.386929 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.381272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:41749 (local address 127.0.0.1:33462)
0810 18:53:48.381440 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.381456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.381535 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.382642 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.382645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.382664 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:48.382780 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.382788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.383853 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.383857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.385282 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.385294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.386615 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.386617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.386691 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.386713 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.386770 (+ 57us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.386833 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 18:53:48.387126 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.381592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.30.153.130:41749)
0810 18:53:48.382086 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.382099 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.382124 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:53:48.382213 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.382218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.382482 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.382673 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:48.382922 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.383727 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.385452 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.386492 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.386634 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.386796 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.386854 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.386954 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":150,"threads_started":1}
I20260810 18:53:48.390405 9501 tablet_service.cc:1467] Processing CreateTablet for tablet 73899f58072b45d4a72f39735759a71a (DEFAULT_TABLE table=test-table [id=e800b4087b4943dc9235e7c7f651d95d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:48.391680 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73899f58072b45d4a72f39735759a71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:48.396839 9837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.382066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37918 (local address 127.30.153.131:39721)
0810 18:53:48.391228 (+ 9162us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.391241 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.391264 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:53:48.391363 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.391369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.391554 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.392328 (+ 774us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:48.392467 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.393525 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.395303 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.396090 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.396244 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.396393 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.396537 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.396647 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9055,"thread_start_us":110,"threads_started":1}
I20260810 18:53:48.397393 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.381901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:39721 (local address 127.0.0.1:37918)
0810 18:53:48.382896 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.382910 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.382993 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.391719 (+ 8726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.391723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.391745 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:48.391846 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.391852 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.393659 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.393663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.395161 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.395173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.396229 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.396233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.396388 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.396424 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.396508 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.397229 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":28,"thread_start_us":103,"threads_started":1}
I20260810 18:53:48.398252 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.382089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46261 (local address 127.0.0.1:33224)
0810 18:53:48.387113 (+ 5024us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.387127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.387197 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.392145 (+ 4948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.392148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.392168 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:48.392247 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.392253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.395209 (+ 2956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.395214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.396364 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.396375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.397856 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.397860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.397950 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.397985 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.398043 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.398123 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4971,"mutex_wait_us":27,"thread_start_us":104,"threads_started":1}
I20260810 18:53:48.398991 9838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.382522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.30.153.129:46261)
0810 18:53:48.390674 (+ 8152us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.390681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.390707 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:53:48.390788 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.390793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.391870 (+ 1077us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.393721 (+ 1851us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:48.393871 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.394977 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.396519 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.397711 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.398569 (+ 858us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.398706 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.398756 (+ 50us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.398846 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8028,"thread_start_us":85,"threads_started":1}
I20260810 18:53:48.400485 9840 tablet_bootstrap.cc:492] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Bootstrap starting.
I20260810 18:53:48.400339 9696 tablet_service.cc:1467] Processing CreateTablet for tablet 73899f58072b45d4a72f39735759a71a (DEFAULT_TABLE table=test-table [id=e800b4087b4943dc9235e7c7f651d95d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:48.401500 9696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73899f58072b45d4a72f39735759a71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:48.401850 9298 tablet_service.cc:1467] Processing CreateTablet for tablet 73899f58072b45d4a72f39735759a71a (DEFAULT_TABLE table=test-table [id=e800b4087b4943dc9235e7c7f651d95d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:48.402772 9840 tablet_bootstrap.cc:654] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:48.402990 9298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73899f58072b45d4a72f39735759a71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:48.403569 9840 log.cc:824] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:48.405252 9840 tablet_bootstrap.cc:492] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: No bootstrap required, opened a new log
I20260810 18:53:48.405395 9840 ts_tablet_manager.cc:1397] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:53:48.409264 9840 raft_consensus.cc:348] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.409543 9840 raft_consensus.cc:374] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:48.409614 9840 raft_consensus.cc:729] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a684fc596e4446378f25533f4a960da7, State: Initialized, Role: FOLLOWER
I20260810 18:53:48.410116 9840 consensus_queue.cc:260] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.411212 9840 ts_tablet_manager.cc:1428] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260810 18:53:48.411532 9594 heartbeater.cc:503] Master 127.30.153.190:40429 was elected leader, sending a full tablet report...
I20260810 18:53:48.412624 9845 tablet_bootstrap.cc:492] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Bootstrap starting.
I20260810 18:53:48.415040 9845 tablet_bootstrap.cc:654] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:48.415915 9845 log.cc:824] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:48.416568 9846 tablet_bootstrap.cc:492] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Bootstrap starting.
I20260810 18:53:48.418284 9845 tablet_bootstrap.cc:492] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: No bootstrap required, opened a new log
I20260810 18:53:48.418555 9845 ts_tablet_manager.cc:1397] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:53:48.418879 9846 tablet_bootstrap.cc:654] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:48.419708 9846 log.cc:824] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:48.421857 9846 tablet_bootstrap.cc:492] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: No bootstrap required, opened a new log
I20260810 18:53:48.422080 9846 ts_tablet_manager.cc:1397] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:53:48.423283 9845 raft_consensus.cc:348] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.423563 9845 raft_consensus.cc:374] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:48.423627 9845 raft_consensus.cc:729] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaae3ac34bf41d9857c39d16129f6c9, State: Initialized, Role: FOLLOWER
I20260810 18:53:48.424125 9845 consensus_queue.cc:260] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.425246 9845 ts_tablet_manager.cc:1428] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260810 18:53:48.425894 9795 heartbeater.cc:503] Master 127.30.153.190:40429 was elected leader, sending a full tablet report...
I20260810 18:53:48.426221 9846 raft_consensus.cc:348] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.426529 9846 raft_consensus.cc:374] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:48.426607 9846 raft_consensus.cc:729] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3388724472a644d38545ea985f6fb264, State: Initialized, Role: FOLLOWER
I20260810 18:53:48.427093 9846 consensus_queue.cc:260] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.428077 9846 ts_tablet_manager.cc:1428] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:53:48.429384 9394 heartbeater.cc:503] Master 127.30.153.190:40429 was elected leader, sending a full tablet report...
W20260810 18:53:48.584513 9797 tablet.cc:2369] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:48.597568 9398 tablet.cc:2369] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:48.598658 9597 tablet.cc:2369] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:48.683293 9851 raft_consensus.cc:482] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:48.683588 9851 raft_consensus.cc:504] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.684847 9851 leader_election.cc:288] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a684fc596e4446378f25533f4a960da7 (127.30.153.130:41749), 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261)
I20260810 18:53:48.688835 9838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.685404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:37163 (local address 127.30.153.129:46261)
0810 18:53:48.685543 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.685550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.685566 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:48.685625 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.685627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.685666 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.685762 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:48.686032 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.686608 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.687715 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.688116 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.688254 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.688313 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.688611 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.688687 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:48.688835 9802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.685230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46261 (local address 127.30.153.131:37163)
0810 18:53:48.685375 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.685390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.685477 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.685779 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.685782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.685798 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:48.685912 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.685917 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.686716 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.686722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.687585 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.687595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.688240 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.688244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.688465 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.688501 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.688593 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.688687 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":141}
I20260810 18:53:48.688836 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.685226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:60385 (local address 127.30.153.130:41749)
0810 18:53:48.685481 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.685485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.685498 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:48.685558 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.685560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.685600 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.685691 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:48.685976 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.686568 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.687746 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.688144 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.688273 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.688326 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.688617 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.688698 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260810 18:53:48.688853 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.685076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:41749 (local address 127.30.153.131:60385)
0810 18:53:48.685226 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:48.685242 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:48.685344 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:48.685709 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:48.685712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:48.685728 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:48.685853 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.685859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.686684 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.686689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.687617 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:48.687627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.688265 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.688268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:48.688463 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.688494 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:53:48.688589 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 18:53:48.688702 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":77}
I20260810 18:53:48.689615 9527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73899f58072b45d4a72f39735759a71a" candidate_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a684fc596e4446378f25533f4a960da7" is_pre_election: true
I20260810 18:53:48.689667 9329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73899f58072b45d4a72f39735759a71a" candidate_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3388724472a644d38545ea985f6fb264" is_pre_election: true
I20260810 18:53:48.689950 9527 raft_consensus.cc:2454] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaae3ac34bf41d9857c39d16129f6c9 in term 0.
I20260810 18:53:48.689958 9329 raft_consensus.cc:2454] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ceaae3ac34bf41d9857c39d16129f6c9 in term 0.
I20260810 18:53:48.690485 9626 leader_election.cc:302] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3388724472a644d38545ea985f6fb264, ceaae3ac34bf41d9857c39d16129f6c9; no voters:
I20260810 18:53:48.690734 9851 raft_consensus.cc:2781] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:48.690847 9851 raft_consensus.cc:482] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:48.690908 9851 raft_consensus.cc:3037] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:48.691991 9851 raft_consensus.cc:504] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.692405 9851 leader_election.cc:288] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [CANDIDATE]: Term 1 election: Requested vote from peers a684fc596e4446378f25533f4a960da7 (127.30.153.130:41749), 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261)
I20260810 18:53:48.692818 9527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73899f58072b45d4a72f39735759a71a" candidate_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a684fc596e4446378f25533f4a960da7"
I20260810 18:53:48.692974 9527 raft_consensus.cc:3037] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:48.692972 9329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73899f58072b45d4a72f39735759a71a" candidate_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3388724472a644d38545ea985f6fb264"
I20260810 18:53:48.693150 9329 raft_consensus.cc:3037] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:48.694245 9527 raft_consensus.cc:2454] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaae3ac34bf41d9857c39d16129f6c9 in term 1.
I20260810 18:53:48.694607 9629 leader_election.cc:302] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a684fc596e4446378f25533f4a960da7, ceaae3ac34bf41d9857c39d16129f6c9; no voters:
I20260810 18:53:48.694772 9329 raft_consensus.cc:2454] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ceaae3ac34bf41d9857c39d16129f6c9 in term 1.
I20260810 18:53:48.694789 9851 raft_consensus.cc:2781] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:48.695024 9851 raft_consensus.cc:686] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 LEADER]: Becoming Leader. State: Replica: ceaae3ac34bf41d9857c39d16129f6c9, State: Running, Role: LEADER
I20260810 18:53:48.695392 9851 consensus_queue.cc:237] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:48.699024 8028 catalog_manager.cc:5495] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ceaae3ac34bf41d9857c39d16129f6c9 (127.30.153.131). New cstate: current_term: 1 leader_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } health_report { overall_health: HEALTHY } } }
I20260810 18:53:48.751497 9837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.747939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37922 (local address 127.30.153.131:39721)
0810 18:53:48.748087 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.748091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.748107 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:48.748215 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.748217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.748275 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.748369 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:48.748588 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.749120 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.749989 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.750196 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.750273 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.750615 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:48.751069 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:48.751169 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.751258 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.751310 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:53:48.757771 9527 raft_consensus.cc:1261] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 1 FOLLOWER]: Refusing update from remote peer ceaae3ac34bf41d9857c39d16129f6c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:48.757771 9329 raft_consensus.cc:1261] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 1 FOLLOWER]: Refusing update from remote peer ceaae3ac34bf41d9857c39d16129f6c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:48.758610 9854 consensus_queue.cc:1035] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:48.758786 9851 consensus_queue.cc:1035] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:48.768483 9859 mvcc.cc:205] Tried to move back new op lower bound from 7317045365784240128 to 7317045365539033088. Current Snapshot: MvccSnapshot[applied={T|T < 7317045365784240128}]
I20260810 18:53:48.773569 9858 mvcc.cc:205] Tried to move back new op lower bound from 7317045365784240128 to 7317045365539033088. Current Snapshot: MvccSnapshot[applied={T|T < 7317045365784240128}]
I20260810 18:53:48.776916 8028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:48.777092 8028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:48.783824 8028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa3fb5cc172547019da531d495dcef0a: Sending DeleteTablet for 3 replicas of tablet 73899f58072b45d4a72f39735759a71a
I20260810 18:53:48.784720 9696 tablet_service.cc:1514] Processing DeleteTablet for tablet 73899f58072b45d4a72f39735759a71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:48 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:37918
I20260810 18:53:48.784792 9501 tablet_service.cc:1514] Processing DeleteTablet for tablet 73899f58072b45d4a72f39735759a71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:48 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33462
I20260810 18:53:48.785265 9866 tablet_replica.cc:330] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: stopping tablet replica
I20260810 18:53:48.785295 9868 tablet_replica.cc:330] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: stopping tablet replica
I20260810 18:53:48.785300 9298 tablet_service.cc:1514] Processing DeleteTablet for tablet 73899f58072b45d4a72f39735759a71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:48 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33224
I20260810 18:53:48.785640 9866 raft_consensus.cc:2229] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:48.785637 9868 raft_consensus.cc:2229] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:48.785981 9868 raft_consensus.cc:2258] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:48.786027 9866 raft_consensus.cc:2258] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:48.787636 9868 ts_tablet_manager.cc:1905] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:48.788586 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 7989
I20260810 18:53:48.792870 9868 ts_tablet_manager.cc:1918] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:48.792964 9868 log.cc:1196] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/73899f58072b45d4a72f39735759a71a
I20260810 18:53:48.793257 9868 ts_tablet_manager.cc:1939] T 73899f58072b45d4a72f39735759a71a P a684fc596e4446378f25533f4a960da7: Deleting consensus metadata
I20260810 18:53:48.794523 9869 tablet_replica.cc:330] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: stopping tablet replica
I20260810 18:53:48.795109 9869 raft_consensus.cc:2229] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:48.795423 9869 raft_consensus.cc:2258] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:48.797694 9869 ts_tablet_manager.cc:1905] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:48.800799 9869 ts_tablet_manager.cc:1918] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:48.800937 9869 log.cc:1196] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/73899f58072b45d4a72f39735759a71a
I20260810 18:53:48.801255 9869 ts_tablet_manager.cc:1939] T 73899f58072b45d4a72f39735759a71a P 3388724472a644d38545ea985f6fb264: Deleting consensus metadata
I20260810 18:53:48.804111 9866 ts_tablet_manager.cc:1905] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:48.807003 9866 ts_tablet_manager.cc:1918] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:48.807202 9866 log.cc:1196] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/73899f58072b45d4a72f39735759a71a
I20260810 18:53:48.807675 9866 ts_tablet_manager.cc:1939] T 73899f58072b45d4a72f39735759a71a P ceaae3ac34bf41d9857c39d16129f6c9: Deleting consensus metadata
W20260810 18:53:48.810294 9626 connection.cc:535] server connection from 127.0.0.1:37918 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 18:53:48.810336 9236 connection.cc:535] server connection from 127.0.0.1:33224 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 18:53:48.810781 9436 connection.cc:535] server connection from 127.0.0.1:33462 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:53:48.812883 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.787983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.30.153.189:39603)
0810 18:53:48.788106 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.788112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.802927 (+ 14815us) server_negotiation.cc:407] Connection header received
0810 18:53:48.803190 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.803193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.803255 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.803326 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.804505 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.805292 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.806542 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.806898 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.806984 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.809021 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.809028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.809034 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.809080 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.810893 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.811383 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.811387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.811389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.811476 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.811795 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.811798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.811800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.812012 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.812148 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.812579 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.812714 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:53:48.823408 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.788211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33476 (local address 127.30.153.188:35207)
0810 18:53:48.788324 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.788329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.812902 (+ 24573us) server_negotiation.cc:407] Connection header received
0810 18:53:48.813417 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.813421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.813483 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.813537 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.814493 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.815296 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.816341 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.816586 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.816704 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.818973 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.818978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.818982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.819019 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.821251 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.821752 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.821756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.821758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.821845 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.822145 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.822149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.822150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.822485 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.822598 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.822921 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.823201 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:53:48.828965 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.818902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.30.153.188:35207)
0810 18:53:48.819486 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.819490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.819500 (+ 10us) server_negotiation.cc:407] Connection header received
0810 18:53:48.819529 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.819531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.819575 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.819669 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.820488 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.821269 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.822484 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.822818 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.822893 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.825332 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.825336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.825339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.825365 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.827131 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.827639 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.827643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.827645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.827729 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.828028 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.828032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.828033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.828270 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.828416 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.828664 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.828787 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502}
I20260810 18:53:48.829104 9043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:48.814964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33420 (local address 127.30.153.189:39603)
0810 18:53:48.817521 (+ 2557us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:48.817538 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:48.817551 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:48.817589 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:48.817591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:48.817635 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:48.817716 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:48.819374 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.820152 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.821834 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:48.822168 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:48.822245 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:48.824404 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:48.824408 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:48.824410 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:48.824436 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:48.826455 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.827703 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.827707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.827709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.827816 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:48.828142 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:48.828145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:48.828147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:48.828353 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:48.828447 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:48.828807 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 18:53:48.828929 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2478}
I20260810 18:53:49.774739 9881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:49.774171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:44465)
0810 18:53:49.774494 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:49.774618 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":102,"threads_started":1}
I20260810 18:53:49.775105 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:49.774674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:37163)
0810 18:53:49.774935 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:49.775026 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":79,"threads_started":1}
W20260810 18:53:49.775137 9394 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:40429 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
W20260810 18:53:49.775461 9594 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:40429 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
I20260810 18:53:49.775951 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:49.775587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:54511)
0810 18:53:49.775793 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:49.775875 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
W20260810 18:53:49.776214 9795 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:40429 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
I20260810 18:53:50.282956 9885 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa3fb5cc172547019da531d495dcef0a)
I20260810 18:53:50.283115 9886 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aa3fb5cc172547019da531d495dcef0a)
I20260810 18:53:50.283241 9885 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } }
I20260810 18:53:50.283381 9886 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } }
I20260810 18:53:50.284410 9886 leader_election.cc:288] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429), c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188:35207)
I20260810 18:53:50.284585 9885 leader_election.cc:288] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429), c2ebd8ba15df4bac8d4e882ce5aac624 (127.30.153.189:39603)
I20260810 18:53:50.285116 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.284563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:48264)
0810 18:53:50.284894 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.284980 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":78,"threads_started":1}
I20260810 18:53:50.285184 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.284555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:48274)
0810 18:53:50.284973 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.285076 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":66,"threads_started":1}
W20260810 18:53:50.285277 8852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
W20260810 18:53:50.285346 9117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
I20260810 18:53:50.286278 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.286029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:48286)
0810 18:53:50.286167 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.286213 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 18:53:50.286284 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.286024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:48292)
0810 18:53:50.286165 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.286212 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260810 18:53:50.286391 8852 leader_election.cc:334] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429): Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
W20260810 18:53:50.286439 9117 leader_election.cc:334] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429): Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
I20260810 18:53:50.289093 9894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.284950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:35207 (local address 127.0.0.1:33498)
0810 18:53:50.285473 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.285489 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:50.285613 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:50.286170 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:50.286174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:50.286192 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:50.286279 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:50.286287 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:50.287177 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:50.287183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:50.287974 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:50.287982 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:50.288680 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:50.288683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:50.288776 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:50.288804 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:53:50.288891 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 18:53:50.288970 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":105,"threads_started":1}
I20260810 18:53:50.289132 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.285243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.30.153.188:35207)
0810 18:53:50.285829 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:50.285833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:50.285848 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:53:50.285895 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:50.285899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:50.285955 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:50.286034 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:50.286445 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:50.287059 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:50.288117 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:50.288555 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:50.288686 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:50.288738 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:50.288929 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 18:53:50.289003 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":73,"threads_started":1}
I20260810 18:53:50.289266 9892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.284648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33432 (local address 127.30.153.189:39603)
0810 18:53:50.284927 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:50.284932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:50.285618 (+ 686us) server_negotiation.cc:407] Connection header received
0810 18:53:50.285855 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:50.285859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:50.285911 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:50.285976 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:50.286595 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:50.287115 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:50.288137 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:50.288654 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:50.288798 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:50.288845 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:50.289030 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 18:53:50.289106 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":59,"threads_started":1}
I20260810 18:53:50.289633 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.284558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:39603 (local address 127.0.0.1:33432)
0810 18:53:50.285525 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.285543 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:50.285695 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:50.286135 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:50.286138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:50.286156 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:50.286250 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:50.286257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:50.287237 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:50.287240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:50.288006 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:50.288013 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:50.288794 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:50.288798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:50.288897 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:50.288916 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 18:53:50.289013 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:53:50.289093 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":146,"thread_start_us":65,"threads_started":1}
I20260810 18:53:50.289740 9151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" is_pre_election: true
I20260810 18:53:50.290017 9151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2ebd8ba15df4bac8d4e882ce5aac624 in term 1.
I20260810 18:53:50.290205 8893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" is_pre_election: true
I20260810 18:53:50.290401 8893 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 in term 1.
I20260810 18:53:50.290400 8856 leader_election.cc:302] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c2ebd8ba15df4bac8d4e882ce5aac624, c4fdc6c4bdfa4aeb9e3d98539b6d0fc9; no voters: aa3fb5cc172547019da531d495dcef0a
I20260810 18:53:50.290735 9886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 18:53:50.290796 9886 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa3fb5cc172547019da531d495dcef0a)
I20260810 18:53:50.290846 9886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:53:50.290828 9118 leader_election.cc:302] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c2ebd8ba15df4bac8d4e882ce5aac624, c4fdc6c4bdfa4aeb9e3d98539b6d0fc9; no voters: aa3fb5cc172547019da531d495dcef0a
I20260810 18:53:50.291188 9885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 18:53:50.291251 9885 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aa3fb5cc172547019da531d495dcef0a)
I20260810 18:53:50.291290 9885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:53:50.292258 9886 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } }
I20260810 18:53:50.292531 9885 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } }
I20260810 18:53:50.292681 9886 leader_election.cc:288] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 2 election: Requested vote from peers aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429), c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188:35207)
I20260810 18:53:50.292956 9885 leader_election.cc:288] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [CANDIDATE]: Term 2 election: Requested vote from peers aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429), c2ebd8ba15df4bac8d4e882ce5aac624 (127.30.153.189:39603)
I20260810 18:53:50.293134 9894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.292857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:48302)
0810 18:53:50.293019 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.293069 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 18:53:50.293293 8893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624"
I20260810 18:53:50.293481 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.293008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:48314)
0810 18:53:50.293370 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.293422 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279}
I20260810 18:53:50.293488 8893 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 in current term 2: Already voted for candidate c2ebd8ba15df4bac8d4e882ce5aac624 in this term.
I20260810 18:53:50.293785 9151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9"
I20260810 18:53:50.293944 9151 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c2ebd8ba15df4bac8d4e882ce5aac624 in current term 2: Already voted for candidate c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 in this term.
I20260810 18:53:50.294207 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.293986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:48316)
0810 18:53:50.294091 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.294147 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260810 18:53:50.294338 9117 leader_election.cc:334] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429): Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
I20260810 18:53:50.294435 9117 leader_election.cc:302] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c4fdc6c4bdfa4aeb9e3d98539b6d0fc9; no voters: aa3fb5cc172547019da531d495dcef0a, c2ebd8ba15df4bac8d4e882ce5aac624
I20260810 18:53:50.294579 9885 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 18:53:50.294852 9894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.294601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:48318)
0810 18:53:50.294752 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.294804 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260810 18:53:50.294956 8852 leader_election.cc:334] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429): Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
I20260810 18:53:50.295012 8852 leader_election.cc:302] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c2ebd8ba15df4bac8d4e882ce5aac624; no voters: aa3fb5cc172547019da531d495dcef0a, c4fdc6c4bdfa4aeb9e3d98539b6d0fc9
I20260810 18:53:50.295163 9886 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 18:53:50.776161 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.775714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:35687)
0810 18:53:50.775994 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.776070 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":113,"threads_started":1}
I20260810 18:53:50.776466 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.775995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:39353)
0810 18:53:50.776312 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.776374 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":90,"threads_started":1}
I20260810 18:53:50.777112 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:50.776770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:40967)
0810 18:53:50.776986 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:50.777035 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
I20260810 18:53:51.777511 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:51.776934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:57151)
0810 18:53:51.777273 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:51.777353 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":82,"threads_started":1}
I20260810 18:53:51.777607 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:51.777126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:59489)
0810 18:53:51.777414 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:51.777486 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260810 18:53:51.778160 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:51.777749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:38259)
0810 18:53:51.777988 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:51.778056 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260810 18:53:52.020772 9907 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:52.020972 9907 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } }
I20260810 18:53:52.021576 9907 leader_election.cc:288] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429), c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188:35207)
I20260810 18:53:52.022100 9151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" is_pre_election: true
I20260810 18:53:52.022287 9151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2ebd8ba15df4bac8d4e882ce5aac624 in term 2.
I20260810 18:53:52.022315 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.021782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:40980)
0810 18:53:52.022028 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.022112 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260810 18:53:52.022694 8856 leader_election.cc:302] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c2ebd8ba15df4bac8d4e882ce5aac624, c4fdc6c4bdfa4aeb9e3d98539b6d0fc9; no voters:
I20260810 18:53:52.022904 9907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 18:53:52.022984 9907 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:52.023029 9907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:53:52.023554 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.023302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:40992)
0810 18:53:52.023424 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.023483 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260810 18:53:52.023660 8852 leader_election.cc:334] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429): Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
I20260810 18:53:52.024546 9907 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } }
I20260810 18:53:52.025010 9907 leader_election.cc:288] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 3 election: Requested vote from peers aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429), c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 (127.30.153.188:35207)
I20260810 18:53:52.025449 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.025137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:41004)
0810 18:53:52.025331 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.025381 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260810 18:53:52.025411 9151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9"
I20260810 18:53:52.025584 9151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:53:52.026947 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.026166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:41018)
0810 18:53:52.026335 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.026865 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 18:53:52.026990 9151 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2ebd8ba15df4bac8d4e882ce5aac624 in term 3.
W20260810 18:53:52.027177 8852 leader_election.cc:334] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429): Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
I20260810 18:53:52.027988 8856 leader_election.cc:302] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c2ebd8ba15df4bac8d4e882ce5aac624, c4fdc6c4bdfa4aeb9e3d98539b6d0fc9; no voters: aa3fb5cc172547019da531d495dcef0a
I20260810 18:53:52.028187 9907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:53:52.028383 9907 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [term 3 LEADER]: Becoming Leader. State: Replica: c2ebd8ba15df4bac8d4e882ce5aac624, State: Running, Role: LEADER
I20260810 18:53:52.028689 9907 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } }
I20260810 18:53:52.029870 9910 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2ebd8ba15df4bac8d4e882ce5aac624. Latest consensus state: current_term: 3 leader_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:52.029992 9910 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:52.030357 9914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:52.033324 9914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:52.033836 9914 catalog_manager.cc:1259] Loaded cluster ID: 5c7346c4c51a452b81bd5ca0dcaa61bc
I20260810 18:53:52.033901 9914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:52.034564 9914 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:52.035115 9914 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Loaded TSK: 0
I20260810 18:53:52.036052 9914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:53:52.057601 8865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:52.063158 8865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:52.067018 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.066750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:41032)
0810 18:53:52.066861 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.066927 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260810 18:53:52.067389 9151 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [term 3 FOLLOWER]: Refusing update from remote peer c2ebd8ba15df4bac8d4e882ce5aac624: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 18:53:52.067793 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.067568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.0.0.1:41042)
0810 18:53:52.067677 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.067727 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 18:53:52.067832 9910 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 18:53:52.068073 8852 consensus_peers.cc:487] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 -> Peer aa3fb5cc172547019da531d495dcef0a (127.30.153.190:40429): Couldn't send request to peer aa3fb5cc172547019da531d495dcef0a. Status: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 18:53:52.069507 9916 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2ebd8ba15df4bac8d4e882ce5aac624. Latest consensus state: current_term: 3 leader_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:52.069670 9916 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:52.070993 9907 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:52.071122 9907 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:52.072021 9916 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "aa3fb5cc172547019da531d495dcef0a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 40429 } } peers { permanent_uuid: "c2ebd8ba15df4bac8d4e882ce5aac624" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 39603 } attrs { promote: false } } peers { permanent_uuid: "c4fdc6c4bdfa4aeb9e3d98539b6d0fc9" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 35207 } attrs { promote: false } } }
I20260810 18:53:52.072131 9916 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4fdc6c4bdfa4aeb9e3d98539b6d0fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:53:52.081482 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.076926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46261 (local address 127.0.0.1:57584)
0810 18:53:52.077041 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.077055 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:52.077141 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:52.077739 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:52.077745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:52.077762 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:52.077875 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.077881 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.079027 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.079030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.080351 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.080362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.081086 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.081089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.081180 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.081207 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:53:52.081263 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 18:53:52.081344 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":73}
I20260810 18:53:52.081872 9933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.077550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:41749 (local address 127.0.0.1:51376)
0810 18:53:52.078221 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.078235 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:52.078324 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:52.078600 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:52.078603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:52.078619 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:52.078708 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.078713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.079606 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.079609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.080544 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.080551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.081491 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.081494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.081599 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.081619 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:53:52.081681 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 18:53:52.081743 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":42,"thread_start_us":88,"threads_started":1}
I20260810 18:53:52.082222 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.077085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57584 (local address 127.30.153.129:46261)
0810 18:53:52.077419 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:52.077427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:52.077444 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:53:52.077499 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:52.077501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:52.077556 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:52.077648 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:52.078016 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.078896 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.080470 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.080952 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.081866 (+ 914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.081987 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:52.082023 (+ 36us) server_negotiation.cc:299] Negotiation successful
0810 18:53:52.082092 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":113,"threads_started":1}
I20260810 18:53:52.082252 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.077970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.30.153.130:41749)
0810 18:53:52.078205 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:52.078209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:52.078289 (+ 80us) server_negotiation.cc:407] Connection header received
0810 18:53:52.078436 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:52.078439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:52.078485 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:52.078591 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:52.078830 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.079495 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.080682 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.081281 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.081386 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.081435 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:52.082017 (+ 582us) server_negotiation.cc:299] Negotiation successful
0810 18:53:52.082105 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":83,"threads_started":1}
I20260810 18:53:52.082871 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.077507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:39721 (local address 127.0.0.1:60148)
0810 18:53:52.077847 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.077862 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:52.077956 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:52.079004 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:52.079007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:52.079026 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:52.079141 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.079147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.080128 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.080131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.081471 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.081479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.082493 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.082497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.082590 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.082612 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:53:52.082691 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:53:52.082753 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":168,"thread_start_us":94,"threads_started":1}
I20260810 18:53:52.082825 9298 tablet_service.cc:1467] Processing CreateTablet for tablet 813b5fb01a5348d4b3ba7775a7a1a045 (DEFAULT_TABLE table=test-table [id=2511ee0214324d48ae4e5988a0d5ee64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:52.082825 9501 tablet_service.cc:1467] Processing CreateTablet for tablet 813b5fb01a5348d4b3ba7775a7a1a045 (DEFAULT_TABLE table=test-table [id=2511ee0214324d48ae4e5988a0d5ee64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:52.082908 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.077736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.30.153.131:39721)
0810 18:53:52.078004 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:52.078776 (+ 772us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:52.078792 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:52.078843 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:52.078846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:52.078894 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:52.078987 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:52.079261 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.080000 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.081574 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.082209 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.082353 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.082407 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:52.082709 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 18:53:52.082782 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260810 18:53:52.083106 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 813b5fb01a5348d4b3ba7775a7a1a045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:52.083106 9298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 813b5fb01a5348d4b3ba7775a7a1a045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:52.083338 9696 tablet_service.cc:1467] Processing CreateTablet for tablet 813b5fb01a5348d4b3ba7775a7a1a045 (DEFAULT_TABLE table=test-table [id=2511ee0214324d48ae4e5988a0d5ee64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:52.083551 9696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 813b5fb01a5348d4b3ba7775a7a1a045. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:52.085733 9937 tablet_bootstrap.cc:492] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264: Bootstrap starting.
I20260810 18:53:52.086375 9940 tablet_bootstrap.cc:492] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: Bootstrap starting.
I20260810 18:53:52.086632 9938 tablet_bootstrap.cc:492] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7: Bootstrap starting.
I20260810 18:53:52.086687 9937 tablet_bootstrap.cc:654] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:52.087352 9940 tablet_bootstrap.cc:654] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:52.087494 9938 tablet_bootstrap.cc:654] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:52.088153 9937 tablet_bootstrap.cc:492] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264: No bootstrap required, opened a new log
I20260810 18:53:52.088268 9937 ts_tablet_manager.cc:1397] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:52.088778 9937 raft_consensus.cc:348] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.088903 9938 tablet_bootstrap.cc:492] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7: No bootstrap required, opened a new log
I20260810 18:53:52.088902 9937 raft_consensus.cc:374] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:52.088943 9937 raft_consensus.cc:729] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3388724472a644d38545ea985f6fb264, State: Initialized, Role: FOLLOWER
I20260810 18:53:52.088981 9938 ts_tablet_manager.cc:1397] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:52.089090 9937 consensus_queue.cc:260] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.089483 9937 ts_tablet_manager.cc:1428] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:52.090081 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.089793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:39237)
0810 18:53:52.089918 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.090006 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260810 18:53:52.090480 9938 raft_consensus.cc:348] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.090610 9938 raft_consensus.cc:374] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:52.090646 9938 raft_consensus.cc:729] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a684fc596e4446378f25533f4a960da7, State: Initialized, Role: FOLLOWER
I20260810 18:53:52.090787 9938 consensus_queue.cc:260] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.090915 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.090632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:40237)
0810 18:53:52.090784 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.090839 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 18:53:52.091172 9938 ts_tablet_manager.cc:1428] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:52.091979 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.091703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:58823)
0810 18:53:52.091857 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.091910 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 18:53:52.092164 9397 heartbeater.cc:503] Master 127.30.153.189:39603 was elected leader, sending a full tablet report...
I20260810 18:53:52.093762 9940 tablet_bootstrap.cc:492] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: No bootstrap required, opened a new log
I20260810 18:53:52.093883 9940 ts_tablet_manager.cc:1397] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260810 18:53:52.094301 9940 raft_consensus.cc:348] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.094393 9940 raft_consensus.cc:374] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:52.094467 9940 raft_consensus.cc:729] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaae3ac34bf41d9857c39d16129f6c9, State: Initialized, Role: FOLLOWER
I20260810 18:53:52.094652 9940 consensus_queue.cc:260] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.094718 9595 heartbeater.cc:503] Master 127.30.153.189:39603 was elected leader, sending a full tablet report...
I20260810 18:53:52.095218 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.094978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:60569)
0810 18:53:52.095096 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.095152 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 18:53:52.095534 9940 ts_tablet_manager.cc:1428] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:52.096496 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.096247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:51047)
0810 18:53:52.096374 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.096429 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 18:53:52.097494 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.097214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:44227)
0810 18:53:52.097366 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.097419 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 18:53:52.097800 9796 heartbeater.cc:503] Master 127.30.153.189:39603 was elected leader, sending a full tablet report...
I20260810 18:53:52.168463 9946 raft_consensus.cc:482] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:52.168628 9946 raft_consensus.cc:504] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.169792 9946 leader_election.cc:288] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261), ceaae3ac34bf41d9857c39d16129f6c9 (127.30.153.131:39721)
I20260810 18:53:52.174618 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.169989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46261 (local address 127.30.153.130:39645)
0810 18:53:52.170249 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.170267 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:52.170396 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:52.170794 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:52.170796 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:52.170831 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:52.171068 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.171074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.172142 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.172146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.173403 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.173414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.174177 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.174180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.174269 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.174287 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 18:53:52.174383 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:53:52.174479 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":167,"spinlock_wait_cycles":1792}
I20260810 18:53:52.174616 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.170185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:43857 (local address 127.30.153.131:39721)
0810 18:53:52.170317 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:52.170322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:52.170622 (+ 300us) server_negotiation.cc:407] Connection header received
0810 18:53:52.170710 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:52.170716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:52.170771 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:52.170842 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:52.171217 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.172001 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.173527 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.173937 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.174033 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.174111 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:52.174385 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 18:53:52.174485 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:53:52.174654 9931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.170089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:39645 (local address 127.30.153.129:46261)
0810 18:53:52.170300 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:52.170305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:52.170326 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:53:52.170617 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:52.170620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:52.170686 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:52.170766 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:52.171204 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.172001 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.173568 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.173979 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.174134 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.174195 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:52.174445 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 18:53:52.174534 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260810 18:53:52.174647 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.170074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:39721 (local address 127.30.153.130:43857)
0810 18:53:52.170522 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.170534 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:52.170627 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:52.170868 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:52.170870 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:52.170887 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:52.171059 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.171069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.172155 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.172157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.173397 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:52.173407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.174070 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.174077 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:52.174238 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.174270 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:53:52.174333 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 18:53:52.174445 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":91,"thread_start_us":141,"threads_started":1}
I20260810 18:53:52.175249 9329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "813b5fb01a5348d4b3ba7775a7a1a045" candidate_uuid: "a684fc596e4446378f25533f4a960da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3388724472a644d38545ea985f6fb264" is_pre_election: true
I20260810 18:53:52.175285 9721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "813b5fb01a5348d4b3ba7775a7a1a045" candidate_uuid: "a684fc596e4446378f25533f4a960da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" is_pre_election: true
I20260810 18:53:52.175384 9329 raft_consensus.cc:2454] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a684fc596e4446378f25533f4a960da7 in term 0.
I20260810 18:53:52.175556 9721 raft_consensus.cc:2454] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a684fc596e4446378f25533f4a960da7 in term 0.
I20260810 18:53:52.175796 9434 leader_election.cc:302] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3388724472a644d38545ea985f6fb264, a684fc596e4446378f25533f4a960da7; no voters:
I20260810 18:53:52.176083 9946 raft_consensus.cc:2781] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:52.176146 9946 raft_consensus.cc:482] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:52.176173 9946 raft_consensus.cc:3037] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:52.177215 9946 raft_consensus.cc:504] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.177593 9946 leader_election.cc:288] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [CANDIDATE]: Term 1 election: Requested vote from peers 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261), ceaae3ac34bf41d9857c39d16129f6c9 (127.30.153.131:39721)
I20260810 18:53:52.178027 9721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "813b5fb01a5348d4b3ba7775a7a1a045" candidate_uuid: "a684fc596e4446378f25533f4a960da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ceaae3ac34bf41d9857c39d16129f6c9"
I20260810 18:53:52.178082 9329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "813b5fb01a5348d4b3ba7775a7a1a045" candidate_uuid: "a684fc596e4446378f25533f4a960da7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3388724472a644d38545ea985f6fb264"
I20260810 18:53:52.178198 9329 raft_consensus.cc:3037] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:52.178282 9721 raft_consensus.cc:3037] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:52.179232 9329 raft_consensus.cc:2454] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a684fc596e4446378f25533f4a960da7 in term 1.
I20260810 18:53:52.179400 9721 raft_consensus.cc:2454] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a684fc596e4446378f25533f4a960da7 in term 1.
I20260810 18:53:52.179539 9434 leader_election.cc:302] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3388724472a644d38545ea985f6fb264, a684fc596e4446378f25533f4a960da7; no voters:
I20260810 18:53:52.179728 9946 raft_consensus.cc:2781] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:52.179941 9946 raft_consensus.cc:686] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 1 LEADER]: Becoming Leader. State: Replica: a684fc596e4446378f25533f4a960da7, State: Running, Role: LEADER
I20260810 18:53:52.180233 9946 consensus_queue.cc:237] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } }
I20260810 18:53:52.180655 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.180436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:36873)
0810 18:53:52.180534 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.180587 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 18:53:52.182811 8867 catalog_manager.cc:5495] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a684fc596e4446378f25533f4a960da7 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "a684fc596e4446378f25533f4a960da7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a684fc596e4446378f25533f4a960da7" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41749 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:52.205798 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.202188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51380 (local address 127.30.153.130:41749)
0810 18:53:52.202322 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:52.202326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:52.202339 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:53:52.202462 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:52.202468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:52.202560 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:52.202660 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:52.202925 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.203459 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.204332 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:52.204537 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:52.204616 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:52.204982 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:52.205442 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:52.205531 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:52.205628 (+ 97us) server_negotiation.cc:299] Negotiation successful
0810 18:53:52.205680 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:52.208945 9329 raft_consensus.cc:1261] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 1 FOLLOWER]: Refusing update from remote peer a684fc596e4446378f25533f4a960da7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:52.209196 9721 raft_consensus.cc:1261] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 FOLLOWER]: Refusing update from remote peer a684fc596e4446378f25533f4a960da7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:52.209589 9949 consensus_queue.cc:1035] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3388724472a644d38545ea985f6fb264" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:52.209887 9946 consensus_queue.cc:1035] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ceaae3ac34bf41d9857c39d16129f6c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:52.211818 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.211499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:58981)
0810 18:53:52.211644 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.211721 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 18:53:52.213493 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.213131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:45973)
0810 18:53:52.213287 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.213364 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:53:52.216027 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.215733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.130:43189)
0810 18:53:52.215897 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.215953 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 18:53:52.218801 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.218438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.131:32979)
0810 18:53:52.218597 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.218656 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 18:53:52.223106 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.222716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:40429 (local address 127.30.153.129:34975)
0810 18:53:52.222953 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.223018 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:40429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260810 18:53:52.223875 8865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:52.224093 8865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:52.224285 9957 mvcc.cc:205] Tried to move back new op lower bound from 7317045379922550784 to 7317045379813048320. Current Snapshot: MvccSnapshot[applied={T|T < 7317045379922550784}]
I20260810 18:53:52.228595 8865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c2ebd8ba15df4bac8d4e882ce5aac624: Sending DeleteTablet for 3 replicas of tablet 813b5fb01a5348d4b3ba7775a7a1a045
I20260810 18:53:52.229524 9696 tablet_service.cc:1514] Processing DeleteTablet for tablet 813b5fb01a5348d4b3ba7775a7a1a045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:52 UTC) from {username='oryx', principal='oryx/127.30.153.189@KRBTEST.COM'} at 127.0.0.1:60148
I20260810 18:53:52.229564 9298 tablet_service.cc:1514] Processing DeleteTablet for tablet 813b5fb01a5348d4b3ba7775a7a1a045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:52 UTC) from {username='oryx', principal='oryx/127.30.153.189@KRBTEST.COM'} at 127.0.0.1:57584
I20260810 18:53:52.229909 9501 tablet_service.cc:1514] Processing DeleteTablet for tablet 813b5fb01a5348d4b3ba7775a7a1a045 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:52 UTC) from {username='oryx', principal='oryx/127.30.153.189@KRBTEST.COM'} at 127.0.0.1:51376
I20260810 18:53:52.229941 9966 tablet_replica.cc:330] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: stopping tablet replica
I20260810 18:53:52.230078 9966 raft_consensus.cc:2229] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:52.230211 9966 raft_consensus.cc:2258] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:52.230511 9967 tablet_replica.cc:330] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264: stopping tablet replica
I20260810 18:53:52.230648 9967 raft_consensus.cc:2229] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:52.230803 9967 raft_consensus.cc:2258] T 813b5fb01a5348d4b3ba7775a7a1a045 P 3388724472a644d38545ea985f6fb264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:52.231237 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 9076
I20260810 18:53:52.240312 9966 ts_tablet_manager.cc:1905] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:52.243274 9966 ts_tablet_manager.cc:1918] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:52.243373 9966 log.cc:1196] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/813b5fb01a5348d4b3ba7775a7a1a045
I20260810 18:53:52.243712 9966 ts_tablet_manager.cc:1939] T 813b5fb01a5348d4b3ba7775a7a1a045 P ceaae3ac34bf41d9857c39d16129f6c9: Deleting consensus metadata
I20260810 18:53:52.244551 9968 tablet_replica.cc:330] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7: stopping tablet replica
I20260810 18:53:52.244582 8856 catalog_manager.cc:4841] TS ceaae3ac34bf41d9857c39d16129f6c9 (127.30.153.131:39721): tablet 813b5fb01a5348d4b3ba7775a7a1a045 (table test-table [id=2511ee0214324d48ae4e5988a0d5ee64]) successfully deleted
I20260810 18:53:52.244699 9968 raft_consensus.cc:2229] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:52.245106 9968 raft_consensus.cc:2258] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:52.245821 9968 ts_tablet_manager.cc:1905] T 813b5fb01a5348d4b3ba7775a7a1a045 P a684fc596e4446378f25533f4a960da7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:52.247524 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.247223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46261 (local address 127.0.0.1:57592)
0810 18:53:52.247365 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.247424 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:46261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 18:53:52.247558 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 9404
W20260810 18:53:52.250689 8852 catalog_manager.cc:4584] TS 3388724472a644d38545ea985f6fb264 (127.30.153.129:46261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 813b5fb01a5348d4b3ba7775a7a1a045: Network error: Client connection negotiation failed: client connection to 127.30.153.129:46261: connect: Connection refused (error 111)
I20260810 18:53:52.260156 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 9602
I20260810 18:53:52.260385 9934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:52.260055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:41749 (local address 127.0.0.1:51396)
0810 18:53:52.260200 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:52.260281 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.130:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:53:52.276135 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 8661
I20260810 18:53:52.290807 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 8822
2026-08-10T18:53:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7703 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:52 dist-test-slave-68xk krb5kdc[9971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:52 dist-test-slave-68xk krb5kdc[9971](info): set up 2 sockets
Aug 10 18:53:52 dist-test-slave-68xk krb5kdc[9971](info): commencing operation
krb5kdc: starting...
W20260810 18:53:53.618249 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:53 dist-test-slave-68xk krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.190.keytab.
Entry for principal oryx/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.190.keytab.
I20260810 18:53:53.749677 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:37787
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.190.keytab
--principal=oryx/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:37787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:53.865371 9988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:53.865691 9988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:53.865779 9988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:53.869858 9988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:53.869956 9988 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 18:53:53.869989 9988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:53.870033 9988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:53.870074 9988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:53.870107 9988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:53.875478 9988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:37787
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.190.keytab
--principal=oryx/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:37787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:53.876575 9988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:53.877615 9988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:53:53.886727 9988 server_base.cc:1034] running on GCE node
W20260810 18:53:53.886783 9994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:53.886759 9997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:53.886893 9995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:53.887562 9988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:53.888676 9988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:53.889920 9988 hybrid_clock.cc:648] HybridClock initialized: now 1786388033889891 us; error 43 us; skew 500 ppm
Aug 10 18:53:53 dist-test-slave-68xk krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388033, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:53.893204 9988 init.cc:377] Logged in from keytab as oryx/127.30.153.190@KRBTEST.COM (short username oryx)
I20260810 18:53:53.894184 9988 webserver.cc:460] Webserver started at http://127.30.153.190:39049/ using document root <none> and password file <none>
I20260810 18:53:53.894860 9988 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:53.894965 9988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:53.895219 9988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:53.897426 9988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "7e77163c28e84b35932172e0a58784e9"
format_stamp: "Formatted at 2026-08-10 18:53:53 on dist-test-slave-68xk"
server_key: "c521f7839ece91fda50e52446deffdbd"
server_key_iv: "880a637086f7ebf34ce97fc937da7d66"
server_key_version: "encryptionkey@0"
I20260810 18:53:53.898026 9988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "7e77163c28e84b35932172e0a58784e9"
format_stamp: "Formatted at 2026-08-10 18:53:53 on dist-test-slave-68xk"
server_key: "c521f7839ece91fda50e52446deffdbd"
server_key_iv: "880a637086f7ebf34ce97fc937da7d66"
server_key_version: "encryptionkey@0"
I20260810 18:53:53.902001 9988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:53.904875 10012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:53.906351 9988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:53.906522 9988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "7e77163c28e84b35932172e0a58784e9"
format_stamp: "Formatted at 2026-08-10 18:53:53 on dist-test-slave-68xk"
server_key: "c521f7839ece91fda50e52446deffdbd"
server_key_iv: "880a637086f7ebf34ce97fc937da7d66"
server_key_version: "encryptionkey@0"
I20260810 18:53:53.906641 9988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:53.918251 9988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:53.921423 9988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:53.921664 9988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:53.930745 9988 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:37787
I20260810 18:53:53.930779 10099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:37787 every 8 connection(s)
I20260810 18:53:53.931792 9988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:53.934880 10102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:53.936669 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 9988
I20260810 18:53:53.936781 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:53.937139 31334 external_mini_cluster.cc:1428] Setting key ef0bdda9b4e4bbd78f24786e47c5d797
I20260810 18:53:53.942116 10102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: Bootstrap starting.
I20260810 18:53:53.944962 10102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:53.945962 10102 log.cc:824] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:53:53 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:53.948323 10102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: No bootstrap required, opened a new log
I20260810 18:53:53.952751 10102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e77163c28e84b35932172e0a58784e9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37787 } }
I20260810 18:53:53.953109 10102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:53.953181 10102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e77163c28e84b35932172e0a58784e9, State: Initialized, Role: FOLLOWER
I20260810 18:53:53.953681 10102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e77163c28e84b35932172e0a58784e9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37787 } }
I20260810 18:53:53.953816 10102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:53.953905 10102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:53.954016 10102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:53.954746 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:53.938779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36948 (local address 127.30.153.190:37787)
0810 18:53:53.939246 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:53.939266 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:53.939309 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:53:53.940156 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:53.940183 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:53.940621 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:53.941008 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:53.941915 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:53.943402 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:53.944387 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:53.945117 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:53.945305 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:53.948323 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:53.948330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:53.948349 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:53.948385 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:53.951274 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:53.951796 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:53.951805 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:53.951812 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:53.951958 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:53.952290 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:53.952294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:53.952295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:53.952837 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:53.953047 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:53.953384 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 18:53:53.953670 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":136,"threads_started":1}
I20260810 18:53:53.955188 10102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e77163c28e84b35932172e0a58784e9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37787 } }
I20260810 18:53:53.955541 10102 leader_election.cc:302] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e77163c28e84b35932172e0a58784e9; no voters:
I20260810 18:53:53.955935 10102 leader_election.cc:288] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:53.956056 10107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:53.956313 10107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [term 1 LEADER]: Becoming Leader. State: Replica: 7e77163c28e84b35932172e0a58784e9, State: Running, Role: LEADER
I20260810 18:53:53.956651 10107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e77163c28e84b35932172e0a58784e9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37787 } }
I20260810 18:53:53.957340 10102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:53.957919 10107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e77163c28e84b35932172e0a58784e9. Latest consensus state: current_term: 1 leader_uuid: "7e77163c28e84b35932172e0a58784e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e77163c28e84b35932172e0a58784e9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37787 } } }
I20260810 18:53:53.958046 10107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:53.958469 10109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e77163c28e84b35932172e0a58784e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e77163c28e84b35932172e0a58784e9" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37787 } } }
I20260810 18:53:53.958585 10109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:53.958742 10114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:53.961807 10114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:53.967557 10114 catalog_manager.cc:1347] Generated new cluster ID: 9d47a31a9f4745d4a76ef25dcdc8ed0e
I20260810 18:53:53.967653 10114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:53.978010 10114 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:53.979839 10114 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:53.995569 10114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: Generated new TSK 0
I20260810 18:53:53.996327 10114 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab.
Entry for principal oryx/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab.
I20260810 18:53:54.033082 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37787
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab
--principal=oryx/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:54.153680 10136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:54.153955 10136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:54.154006 10136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:54.158540 10136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:54.158653 10136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:54.158711 10136 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:54.158757 10136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:54.164587 10136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab
--principal=oryx/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:37787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:54.165709 10136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:54.166795 10136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:54.177361 10143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:54.177349 10144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:54.177349 10147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:54.177747 10136 server_base.cc:1034] running on GCE node
I20260810 18:53:54.178161 10136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:54.178916 10136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:54.180154 10136 hybrid_clock.cc:648] HybridClock initialized: now 1786388034180125 us; error 56 us; skew 500 ppm
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:54.183048 10136 init.cc:377] Logged in from keytab as oryx/127.30.153.129@KRBTEST.COM (short username oryx)
I20260810 18:53:54.183964 10136 webserver.cc:460] Webserver started at http://127.30.153.129:37243/ using document root <none> and password file <none>
I20260810 18:53:54.184535 10136 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:54.184631 10136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:54.184867 10136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:54.187021 10136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "2db36344207b4d1097f1be9df43e70a2"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "e9e2dc805726ec4ad9377fe3fa7ff325"
server_key_iv: "ab12099f378cc082e9c86b649ab494d8"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.187597 10136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "2db36344207b4d1097f1be9df43e70a2"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "e9e2dc805726ec4ad9377fe3fa7ff325"
server_key_iv: "ab12099f378cc082e9c86b649ab494d8"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.191466 10136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:53:54.194101 10157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:54.195468 10136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:54.195612 10136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "2db36344207b4d1097f1be9df43e70a2"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "e9e2dc805726ec4ad9377fe3fa7ff325"
server_key_iv: "ab12099f378cc082e9c86b649ab494d8"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.195735 10136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:54.215114 10136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:54.218076 10136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:54.218297 10136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:54.219859 10136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:54.219935 10136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:54.220006 10136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:54.220055 10136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:54.231447 10136 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:33411
I20260810 18:53:54.231552 10320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:33411 every 8 connection(s)
I20260810 18:53:54.232584 10136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:54.235528 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.220853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:33315 (local address 127.30.153.190:37787)
0810 18:53:54.220988 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.220992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.222048 (+ 1056us) server_negotiation.cc:407] Connection header received
0810 18:53:54.223173 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.223178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.223252 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.223319 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:54.225189 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.226055 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.227986 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.228332 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.228420 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.231740 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:54.231744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:54.231747 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:54.231776 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:54.233546 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.234018 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.234021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.234022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.234101 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.234495 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.234498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.234500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.234687 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:54.234765 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.235266 (+ 501us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.235393 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:53:54.236220 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.221404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37787 (local address 127.30.153.129:33315)
0810 18:53:54.221897 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.221940 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.222875 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.223821 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.223832 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.224307 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:54.224994 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.225017 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.226222 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.226230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.227844 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.227857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.228493 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.228498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.228729 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.229488 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:54.229514 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:54.231548 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:54.233654 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.233659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.233669 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.233921 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.234223 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.234225 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.234227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.234357 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.234777 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:54.234782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:54.234995 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.235151 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.235387 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":142,"threads_started":1}
I20260810 18:53:54.237517 10321 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37787
I20260810 18:53:54.237659 10321 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:54.238253 10321 heartbeater.cc:511] Master 127.30.153.190:37787 requested a full tablet report, sending...
I20260810 18:53:54.239980 10031 ts_manager.cc:194] Registered new tserver with Master: 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411)
I20260810 18:53:54.240835 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 10136
I20260810 18:53:54.240980 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:54.241319 31334 external_mini_cluster.cc:1428] Setting key c3c8f6aa7d0cc660f31d55c9d055d90f
I20260810 18:53:54.241541 10031 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:33315
I20260810 18:53:54.248389 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.239953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37787 (local address 127.30.153.129:43365)
0810 18:53:54.240309 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.240323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.240422 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.240655 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.240658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.240864 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:54.241020 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.241031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.241962 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.241966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.242838 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.242850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.243644 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.243650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.243779 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.244307 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:54.244328 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:54.244805 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:54.246840 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.246842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.246843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.247034 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.247351 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.247353 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.247354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.247428 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.247869 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:54.247871 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:54.247944 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.248026 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.248157 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":123,"threads_started":1}
I20260810 18:53:54.248484 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.240034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:43365 (local address 127.30.153.190:37787)
0810 18:53:54.240154 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.240158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.240405 (+ 247us) server_negotiation.cc:407] Connection header received
0810 18:53:54.240501 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.240503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.240540 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.240635 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:54.241169 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.241790 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.242982 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.243364 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.243457 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.244962 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:54.244969 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:54.244974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:54.245007 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:54.246730 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.247151 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.247157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.247162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.247254 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.247520 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.247526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.247530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.247771 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:54.247886 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.248131 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.248269 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for oryx/127.30.153.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab.
Entry for principal oryx/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab.
I20260810 18:53:54.274351 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37787
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab
--principal=oryx/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:54.388054 10335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:54.388303 10335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:54.388347 10335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:54.392170 10335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:54.392216 10335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:54.392253 10335 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:54.392284 10335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:54.397337 10335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab
--principal=oryx/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:37787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:54.398329 10335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:54.399509 10335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:53:54.409772 10335 server_base.cc:1034] running on GCE node
W20260810 18:53:54.409725 10345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:54.409746 10348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:54.409989 10344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:54.410514 10335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:54.411279 10335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:54.412452 10335 hybrid_clock.cc:648] HybridClock initialized: now 1786388034412424 us; error 39 us; skew 500 ppm
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:54.415299 10335 init.cc:377] Logged in from keytab as oryx/127.30.153.130@KRBTEST.COM (short username oryx)
I20260810 18:53:54.416231 10335 webserver.cc:460] Webserver started at http://127.30.153.130:43397/ using document root <none> and password file <none>
I20260810 18:53:54.416824 10335 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:54.416921 10335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:54.417169 10335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:54.419299 10335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "a9e233e06d7d453eb125f9a5031352af"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "a76d7ea7c2a92de28270688d4a9c544a"
server_key_iv: "e9353964e9db844b37473ce0e967c9f3"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.419876 10335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "a9e233e06d7d453eb125f9a5031352af"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "a76d7ea7c2a92de28270688d4a9c544a"
server_key_iv: "e9353964e9db844b37473ce0e967c9f3"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.423571 10335 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 18:53:54.426152 10360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:54.427402 10335 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:53:54.427536 10335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "a9e233e06d7d453eb125f9a5031352af"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "a76d7ea7c2a92de28270688d4a9c544a"
server_key_iv: "e9353964e9db844b37473ce0e967c9f3"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.427659 10335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:54.447402 10335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:54.450831 10335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:54.451061 10335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:54.452642 10335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:54.452728 10335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:54.452827 10335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:54.452854 10335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:54.464401 10335 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:38053
I20260810 18:53:54.465348 10335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:54.466197 10533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:38053 every 8 connection(s)
I20260810 18:53:54.467674 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.453416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:37737 (local address 127.30.153.190:37787)
0810 18:53:54.453559 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.453563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.454860 (+ 1297us) server_negotiation.cc:407] Connection header received
0810 18:53:54.456067 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.456071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.456136 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.456244 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:54.457445 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.458087 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.459856 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.460079 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.460226 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.463707 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:54.463710 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:54.463712 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:54.463738 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:54.465345 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.466032 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.466035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.466037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.466143 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.466587 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.466590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.466591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.466787 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:54.466870 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.467409 (+ 539us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.467536 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:53:54.468422 10373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.453931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37787 (local address 127.30.153.130:37737)
0810 18:53:54.454642 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.454733 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.455771 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.456484 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.456491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.456829 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:54.457291 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.457304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.458242 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.458265 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.459726 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.459736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.460295 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.460299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.460477 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.461122 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:54.461146 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:54.463505 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:54.465495 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.465500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.465514 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.465896 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.466268 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.466274 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.466278 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.466477 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.466896 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:54.466901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:54.467142 (+ 241us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.467297 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.467553 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":151,"threads_started":1}
I20260810 18:53:54.469671 10534 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37787
I20260810 18:53:54.469810 10534 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:54.470366 10534 heartbeater.cc:511] Master 127.30.153.190:37787 requested a full tablet report, sending...
I20260810 18:53:54.471287 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 10335
I20260810 18:53:54.471424 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:54.471570 10031 ts_manager.cc:194] Registered new tserver with Master: a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053)
I20260810 18:53:54.471753 31334 external_mini_cluster.cc:1428] Setting key 8d47548de88307c8a85a42a760b67e60
I20260810 18:53:54.472343 10031 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:37737
I20260810 18:53:54.479417 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.472380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:38061 (local address 127.30.153.190:37787)
0810 18:53:54.472511 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.472516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.472532 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:53:54.472677 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.472680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.472720 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.472767 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:54.473314 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.473813 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.474790 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.475011 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.475126 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.476218 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:54.476222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:54.476224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:54.476255 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:54.477679 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.478168 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.478171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.478172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.478249 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.478586 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.478589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.478590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.478808 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:54.478891 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.479213 (+ 322us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.479310 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:53:54.479624 10542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.472100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37787 (local address 127.30.153.130:38061)
0810 18:53:54.472442 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.472457 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.472544 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.472892 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.472894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.473097 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:54.473196 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.473202 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.473948 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.473952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.474661 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.474671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.475124 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.475128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.475227 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.475770 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:54.475782 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:54.476083 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:54.477796 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.477802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.477807 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.478069 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.478350 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.478356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.478360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.478492 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.478914 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:54.478920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:54.479033 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.479151 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.479328 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for oryx/127.30.153.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab.
Entry for principal oryx/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab.
I20260810 18:53:54.506233 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37787
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab
--principal=oryx/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:54.620196 10548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:54.620463 10548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:54.620507 10548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:54.624500 10548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:54.624559 10548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:54.624598 10548 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 18:53:54.624634 10548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:54.629730 10548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab
--principal=oryx/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.153.190:37787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:54.630793 10548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:54.631843 10548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:54.641654 10559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:54.641819 10548 server_base.cc:1034] running on GCE node
W20260810 18:53:54.641638 10555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:54.641669 10556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:54.642488 10548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:54.643115 10548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:54.644306 10548 hybrid_clock.cc:648] HybridClock initialized: now 1786388034644277 us; error 42 us; skew 500 ppm
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:54.647284 10548 init.cc:377] Logged in from keytab as oryx/127.30.153.131@KRBTEST.COM (short username oryx)
I20260810 18:53:54.648126 10548 webserver.cc:460] Webserver started at http://127.30.153.131:42459/ using document root <none> and password file <none>
I20260810 18:53:54.648697 10548 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:54.648804 10548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:54.649063 10548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:54.651196 10548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "69f9f60c450b4d43843c81d24b9def4d"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "d80b35069b82a44c5ec67253b2d2514c"
server_key_iv: "2f4900264f2719eaace67f9a36078c7e"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.651791 10548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "69f9f60c450b4d43843c81d24b9def4d"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "d80b35069b82a44c5ec67253b2d2514c"
server_key_iv: "2f4900264f2719eaace67f9a36078c7e"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.655774 10548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260810 18:53:54.658581 10569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:54.659960 10548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:53:54.660192 10548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "69f9f60c450b4d43843c81d24b9def4d"
format_stamp: "Formatted at 2026-08-10 18:53:54 on dist-test-slave-68xk"
server_key: "d80b35069b82a44c5ec67253b2d2514c"
server_key_iv: "2f4900264f2719eaace67f9a36078c7e"
server_key_version: "encryptionkey@0"
I20260810 18:53:54.660323 10548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:54.678769 10548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:54.682230 10548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:54.682502 10548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:54.684002 10548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:54.684085 10548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:54.684171 10548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:54.684206 10548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:54.695508 10548 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:35199
I20260810 18:53:54.695828 10737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:35199 every 8 connection(s)
I20260810 18:53:54.696532 10548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:53:54.698629 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.685012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:38547 (local address 127.30.153.190:37787)
0810 18:53:54.685162 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.685166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.686273 (+ 1107us) server_negotiation.cc:407] Connection header received
0810 18:53:54.687542 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.687546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.687626 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.687733 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:54.689299 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.690184 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.691511 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.691759 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.691879 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.694759 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:54.694763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:54.694765 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:54.694794 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:54.696465 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.697008 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.697013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.697015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.697136 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.697546 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.697553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.697556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.697761 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:54.697848 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.698338 (+ 490us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.698496 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:53:54.699398 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.685589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37787 (local address 127.30.153.131:38547)
0810 18:53:54.686085 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.686158 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.687274 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.687938 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.687948 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.688425 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:54.689128 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.689154 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.690302 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.690305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.691379 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.691390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.691936 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.691941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.692122 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.692724 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:54.692740 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:54.694588 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:54.696590 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.696596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.696617 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.696891 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.697250 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.697253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.697255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.697418 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.697860 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:54.697865 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:54.698077 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.698262 (+ 185us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.698573 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":130,"threads_started":1}
I20260810 18:53:54.700657 10738 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37787
I20260810 18:53:54.700815 10738 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:54.701315 10738 heartbeater.cc:511] Master 127.30.153.190:37787 requested a full tablet report, sending...
I20260810 18:53:54.702482 10031 ts_manager.cc:194] Registered new tserver with Master: 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199)
I20260810 18:53:54.703261 10031 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:38547
I20260810 18:53:54.704041 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 10548
I20260810 18:53:54.704190 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:53:54.704527 31334 external_mini_cluster.cc:1428] Setting key f2211f2cb1a88e6674ec587998f87b66
I20260810 18:53:54.706822 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:53:54.711086 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.703175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:59621 (local address 127.30.153.190:37787)
0810 18:53:54.703297 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.703307 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.703411 (+ 104us) server_negotiation.cc:407] Connection header received
0810 18:53:54.703570 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.703573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.703619 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.703700 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:54.704204 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.704713 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.705801 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.706102 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.706212 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.707576 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:54.707579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:54.707581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:54.707603 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:54.709046 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.709559 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.709564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.709567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.709674 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.710048 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.710053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.710055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.710294 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:54.710440 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.710873 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.710971 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:53:54.711272 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.702994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37787 (local address 127.30.153.131:59621)
0810 18:53:54.703334 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.703351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.703450 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.703738 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.703741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.703964 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:54.704069 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.704074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.704854 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.704857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.705675 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.705685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.706254 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.706257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.706335 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.706996 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:54.707011 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:54.707435 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:54.709198 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.709204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.709208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.709460 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.709844 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:54.709847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:54.709849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:54.709944 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:54.710501 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:54.710504 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:54.710587 (+ 83us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.710786 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.710972 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":1,"thread_start_us":122,"threads_started":1}
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:53:54.734660 10105 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:54.727438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.30.153.190:37787)
0810 18:53:54.727579 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.727582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.727596 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:53:54.727645 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.727648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.727688 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.727759 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:54.728572 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.729115 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.730050 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.730329 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.730513 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.734533 (+ 4020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36960: BlockingRecv error: recv got EOF from 127.0.0.1:36960 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 18:53:54 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388034, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:54.744665 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.736366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36962 (local address 127.30.153.190:37787)
0810 18:53:54.736526 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.736529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.736675 (+ 146us) server_negotiation.cc:407] Connection header received
0810 18:53:54.736760 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.736766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.736819 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.736899 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:54.737507 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.738088 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.739018 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.739277 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.739369 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.741646 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:54.741649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:54.741651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:54.741674 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:54.743175 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.743619 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.743622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.743623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.743706 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:54.743959 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:54.743961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:54.743962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:54.744130 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:54.744237 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.744437 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.744532 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:53:54.747782 10031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:54.750921 10031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:54.765859 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.760439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33888 (local address 127.30.153.130:38053)
0810 18:53:54.760865 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.760879 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.760903 (+ 24us) server_negotiation.cc:407] Connection header received
0810 18:53:54.760983 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.760988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.761161 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.761319 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:54.761505 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.762687 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.764264 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.765105 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.765250 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.765378 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.765592 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.765710 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":137,"threads_started":1}
I20260810 18:53:54.765854 10767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.759656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:33411 (local address 127.0.0.1:37930)
0810 18:53:54.760016 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.760067 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.760236 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.760994 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.760999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.761024 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:54.761177 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.761184 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.762854 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.762857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.764187 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.764198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.765269 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.765272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.765619 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.765641 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.765708 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.765763 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":357,"thread_start_us":76,"threads_started":1}
I20260810 18:53:54.766353 10769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.760270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35199 (local address 127.0.0.1:59050)
0810 18:53:54.761510 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.761522 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.761612 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.762067 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.762070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.762088 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:54.762204 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.762213 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.763498 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.763502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.764618 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.764627 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.765924 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.765928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.766028 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.766052 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.766162 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.766231 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1171,"mutex_wait_us":3,"thread_start_us":94,"threads_started":1}
I20260810 18:53:54.766453 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.760689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59050 (local address 127.30.153.131:35199)
0810 18:53:54.760993 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.761004 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.761716 (+ 712us) server_negotiation.cc:407] Connection header received
0810 18:53:54.761798 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.761809 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.761935 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.762087 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:54.762332 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.763370 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.764742 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.765613 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.765740 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.765853 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.766196 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.766284 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":112,"threads_started":1}
I20260810 18:53:54.765854 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.760078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38053 (local address 127.0.0.1:33888)
0810 18:53:54.760487 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.760501 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.760586 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.761282 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.761285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.761301 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:54.761407 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.761415 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.762810 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.762814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.764122 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.764132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.765228 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.765235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.765386 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.765431 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.765542 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.765671 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":95,"thread_start_us":92,"threads_started":1}
I20260810 18:53:54.766738 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.759820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.30.153.129:33411)
0810 18:53:54.760446 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.760453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.760479 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:53:54.760565 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.760573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.760811 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.761329 (+ 518us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:54.761475 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.762691 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.764329 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.765024 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.765399 (+ 375us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.765536 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.766227 (+ 691us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.766336 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":72,"threads_started":1}
I20260810 18:53:54.769316 10433 tablet_service.cc:1467] Processing CreateTablet for tablet db5c1d735ac049cc922a7afaa264e7cb (DEFAULT_TABLE table=test-table [id=02847a24d01f4ba6b2290dedad40f10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:54.769423 10644 tablet_service.cc:1467] Processing CreateTablet for tablet db5c1d735ac049cc922a7afaa264e7cb (DEFAULT_TABLE table=test-table [id=02847a24d01f4ba6b2290dedad40f10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:54.769397 10229 tablet_service.cc:1467] Processing CreateTablet for tablet db5c1d735ac049cc922a7afaa264e7cb (DEFAULT_TABLE table=test-table [id=02847a24d01f4ba6b2290dedad40f10a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:54.770448 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5c1d735ac049cc922a7afaa264e7cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:54.770442 10229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5c1d735ac049cc922a7afaa264e7cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:54.770464 10433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db5c1d735ac049cc922a7afaa264e7cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:54.777339 10773 tablet_bootstrap.cc:492] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: Bootstrap starting.
I20260810 18:53:54.778263 10775 tablet_bootstrap.cc:492] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: Bootstrap starting.
I20260810 18:53:54.779664 10773 tablet_bootstrap.cc:654] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:54.780241 10775 tablet_bootstrap.cc:654] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:54.780453 10773 log.cc:824] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:54.780974 10775 log.cc:824] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:54.781491 10774 tablet_bootstrap.cc:492] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: Bootstrap starting.
I20260810 18:53:54.782636 10773 tablet_bootstrap.cc:492] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: No bootstrap required, opened a new log
I20260810 18:53:54.782852 10773 ts_tablet_manager.cc:1397] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:53:54.783088 10775 tablet_bootstrap.cc:492] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: No bootstrap required, opened a new log
I20260810 18:53:54.783270 10775 ts_tablet_manager.cc:1397] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260810 18:53:54.783808 10774 tablet_bootstrap.cc:654] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:54.784607 10774 log.cc:824] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:54.786230 10773 raft_consensus.cc:348] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.786489 10773 raft_consensus.cc:374] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:54.786546 10773 raft_consensus.cc:729] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e233e06d7d453eb125f9a5031352af, State: Initialized, Role: FOLLOWER
I20260810 18:53:54.786607 10774 tablet_bootstrap.cc:492] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: No bootstrap required, opened a new log
I20260810 18:53:54.786787 10774 ts_tablet_manager.cc:1397] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:53:54.787041 10773 consensus_queue.cc:260] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.787027 10775 raft_consensus.cc:348] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.787266 10775 raft_consensus.cc:374] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:54.787326 10775 raft_consensus.cc:729] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f9f60c450b4d43843c81d24b9def4d, State: Initialized, Role: FOLLOWER
I20260810 18:53:54.787737 10775 consensus_queue.cc:260] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.788024 10773 ts_tablet_manager.cc:1428] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260810 18:53:54.788522 10534 heartbeater.cc:503] Master 127.30.153.190:37787 was elected leader, sending a full tablet report...
I20260810 18:53:54.788766 10775 ts_tablet_manager.cc:1428] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260810 18:53:54.789101 10738 heartbeater.cc:503] Master 127.30.153.190:37787 was elected leader, sending a full tablet report...
I20260810 18:53:54.791229 10774 raft_consensus.cc:348] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.791493 10774 raft_consensus.cc:374] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:54.791545 10774 raft_consensus.cc:729] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db36344207b4d1097f1be9df43e70a2, State: Initialized, Role: FOLLOWER
I20260810 18:53:54.791915 10774 consensus_queue.cc:260] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.792533 10321 heartbeater.cc:503] Master 127.30.153.190:37787 was elected leader, sending a full tablet report...
I20260810 18:53:54.792840 10774 ts_tablet_manager.cc:1428] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:53:54.883388 10790 raft_consensus.cc:482] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:54.883598 10790 raft_consensus.cc:504] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.884935 10790 leader_election.cc:288] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053)
I20260810 18:53:54.889874 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.885257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38053 (local address 127.30.153.131:58293)
0810 18:53:54.885361 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.885379 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.885506 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.885868 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.885872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.885898 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:54.886056 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.886065 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.887036 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.887042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.888412 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.888424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.889418 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.889421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.889523 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.889550 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.889631 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.889724 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":25}
I20260810 18:53:54.890398 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.885392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:58293 (local address 127.30.153.130:38053)
0810 18:53:54.885558 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.885565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.885582 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:53:54.885657 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.885663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.885726 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.885814 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:54.886192 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.886907 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.888564 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.889191 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.889307 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.889361 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.889693 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.889867 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:53:54.891258 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.885257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:33411 (local address 127.30.153.131:52603)
0810 18:53:54.885957 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:54.885972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:54.886090 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:54.886358 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:54.886360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:54.886377 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:54.887114 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.887128 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.888382 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.888386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.889575 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:54.889587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.890744 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.890749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:54.890870 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.890891 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:53:54.890986 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 18:53:54.891073 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":357,"thread_start_us":144,"threads_started":1}
I20260810 18:53:54.891614 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.885429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:52603 (local address 127.30.153.129:33411)
0810 18:53:54.885562 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.885566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.886081 (+ 515us) server_negotiation.cc:407] Connection header received
0810 18:53:54.886191 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.886194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.886249 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.886335 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:54.887338 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.888232 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.889727 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.890407 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.890534 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.890592 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.891042 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.891451 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:53:54.891911 10470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5c1d735ac049cc922a7afaa264e7cb" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e233e06d7d453eb125f9a5031352af" is_pre_election: true
I20260810 18:53:54.892280 10470 raft_consensus.cc:2454] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 0.
I20260810 18:53:54.892288 10256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5c1d735ac049cc922a7afaa264e7cb" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2" is_pre_election: true
I20260810 18:53:54.892568 10256 raft_consensus.cc:2454] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 0.
I20260810 18:53:54.892889 10574 leader_election.cc:302] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69f9f60c450b4d43843c81d24b9def4d, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:54.896534 10790 raft_consensus.cc:2781] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:54.896648 10790 raft_consensus.cc:482] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:54.896703 10790 raft_consensus.cc:3037] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:54.897972 10790 raft_consensus.cc:504] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.898528 10790 leader_election.cc:288] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 election: Requested vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053)
I20260810 18:53:54.898815 10256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5c1d735ac049cc922a7afaa264e7cb" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2"
I20260810 18:53:54.898941 10470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db5c1d735ac049cc922a7afaa264e7cb" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e233e06d7d453eb125f9a5031352af"
I20260810 18:53:54.898962 10256 raft_consensus.cc:3037] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:54.899096 10470 raft_consensus.cc:3037] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:54.900074 10256 raft_consensus.cc:2454] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 1.
I20260810 18:53:54.900418 10572 leader_election.cc:302] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db36344207b4d1097f1be9df43e70a2, 69f9f60c450b4d43843c81d24b9def4d; no voters:
I20260810 18:53:54.900540 10470 raft_consensus.cc:2454] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 1.
I20260810 18:53:54.900612 10790 raft_consensus.cc:2781] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:54.900853 10790 raft_consensus.cc:686] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 1 LEADER]: Becoming Leader. State: Replica: 69f9f60c450b4d43843c81d24b9def4d, State: Running, Role: LEADER
I20260810 18:53:54.901229 10790 consensus_queue.cc:237] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.904568 10033 catalog_manager.cc:5495] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131). New cstate: current_term: 1 leader_uuid: "69f9f60c450b4d43843c81d24b9def4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } health_report { overall_health: HEALTHY } } }
I20260810 18:53:54.920120 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:54.916410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.30.153.131:35199)
0810 18:53:54.916555 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:54.916559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:54.916573 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:53:54.916645 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:54.916653 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:54.916710 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:54.916802 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:54.917064 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.917596 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.918663 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:54.918887 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:54.918973 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:54.919297 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:54.919742 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:54.919831 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:54.919945 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 18:53:54.920000 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:53:54.926287 10256 raft_consensus.cc:1261] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Refusing update from remote peer 69f9f60c450b4d43843c81d24b9def4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:54.926287 10470 raft_consensus.cc:1261] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Refusing update from remote peer 69f9f60c450b4d43843c81d24b9def4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:54.927101 10790 consensus_queue.cc:1035] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:54.927357 10794 consensus_queue.cc:1035] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:54.934346 10797 mvcc.cc:205] Tried to move back new op lower bound from 7317045391050047488 to 7317045390958329856. Current Snapshot: MvccSnapshot[applied={T|T < 7317045391050047488}]
I20260810 18:53:54.936340 10798 mvcc.cc:205] Tried to move back new op lower bound from 7317045391050047488 to 7317045390958329856. Current Snapshot: MvccSnapshot[applied={T|T < 7317045391050047488}]
I20260810 18:53:54.941918 10800 mvcc.cc:205] Tried to move back new op lower bound from 7317045391050047488 to 7317045390958329856. Current Snapshot: MvccSnapshot[applied={T|T < 7317045391050047488}]
I20260810 18:53:54.945150 10033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:54.945339 10033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:54.947721 10033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: Sending DeleteTablet for 3 replicas of tablet db5c1d735ac049cc922a7afaa264e7cb
I20260810 18:53:54.948539 10644 tablet_service.cc:1514] Processing DeleteTablet for tablet db5c1d735ac049cc922a7afaa264e7cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:54 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:59050
I20260810 18:53:54.948665 10433 tablet_service.cc:1514] Processing DeleteTablet for tablet db5c1d735ac049cc922a7afaa264e7cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:54 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33888
I20260810 18:53:54.948674 10229 tablet_service.cc:1514] Processing DeleteTablet for tablet db5c1d735ac049cc922a7afaa264e7cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:54 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:37930
I20260810 18:53:54.949157 10810 tablet_replica.cc:330] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: stopping tablet replica
I20260810 18:53:54.949463 10810 raft_consensus.cc:2229] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:54.949797 10810 raft_consensus.cc:2258] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:54.949927 10033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:53:54.950060 10811 tablet_replica.cc:330] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: stopping tablet replica
W20260810 18:53:54.950342 10033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:54.950343 10811 raft_consensus.cc:2229] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:54.950654 10811 raft_consensus.cc:2258] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:54.952016 10811 ts_tablet_manager.cc:1905] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:54.952170 10810 ts_tablet_manager.cc:1905] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:54.954602 10812 tablet_replica.cc:330] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: stopping tablet replica
I20260810 18:53:54.954901 10812 raft_consensus.cc:2229] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:54.955459 10229 tablet_service.cc:1467] Processing CreateTablet for tablet 8a9c4789d08347d888b96c1ea4698c70 (DEFAULT_TABLE table=test-table [id=7e01d89f598d4be2a94012bc7e42512b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:54.955662 10644 tablet_service.cc:1467] Processing CreateTablet for tablet 8a9c4789d08347d888b96c1ea4698c70 (DEFAULT_TABLE table=test-table [id=7e01d89f598d4be2a94012bc7e42512b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:54.955736 10229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a9c4789d08347d888b96c1ea4698c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:54.955928 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a9c4789d08347d888b96c1ea4698c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:54.956229 10810 ts_tablet_manager.cc:1918] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:54.956317 10810 log.cc:1196] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/db5c1d735ac049cc922a7afaa264e7cb
I20260810 18:53:54.956629 10810 ts_tablet_manager.cc:1939] T db5c1d735ac049cc922a7afaa264e7cb P a9e233e06d7d453eb125f9a5031352af: Deleting consensus metadata
I20260810 18:53:54.958283 10774 tablet_bootstrap.cc:492] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: Bootstrap starting.
I20260810 18:53:54.958375 10775 tablet_bootstrap.cc:492] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: Bootstrap starting.
I20260810 18:53:54.958791 10016 catalog_manager.cc:4841] TS a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053): tablet db5c1d735ac049cc922a7afaa264e7cb (table test-table [id=02847a24d01f4ba6b2290dedad40f10a]) successfully deleted
I20260810 18:53:54.959419 10774 tablet_bootstrap.cc:654] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:54.959434 10812 raft_consensus.cc:2258] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:54.959465 10775 tablet_bootstrap.cc:654] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:54.960350 10433 tablet_service.cc:1467] Processing CreateTablet for tablet 8a9c4789d08347d888b96c1ea4698c70 (DEFAULT_TABLE table=test-table [id=7e01d89f598d4be2a94012bc7e42512b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:54.960628 10433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a9c4789d08347d888b96c1ea4698c70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:54.963001 10773 tablet_bootstrap.cc:492] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: Bootstrap starting.
I20260810 18:53:54.963068 10812 ts_tablet_manager.cc:1905] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:54.963636 10774 tablet_bootstrap.cc:492] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: No bootstrap required, opened a new log
I20260810 18:53:54.963716 10774 ts_tablet_manager.cc:1397] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260810 18:53:54.963979 10773 tablet_bootstrap.cc:654] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:54.964358 10811 ts_tablet_manager.cc:1918] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:54.964448 10811 log.cc:1196] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/db5c1d735ac049cc922a7afaa264e7cb
I20260810 18:53:54.964547 10774 raft_consensus.cc:348] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.964675 10774 raft_consensus.cc:374] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:54.964711 10774 raft_consensus.cc:729] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db36344207b4d1097f1be9df43e70a2, State: Initialized, Role: FOLLOWER
I20260810 18:53:54.964818 10811 ts_tablet_manager.cc:1939] T db5c1d735ac049cc922a7afaa264e7cb P 2db36344207b4d1097f1be9df43e70a2: Deleting consensus metadata
I20260810 18:53:54.964923 10774 consensus_queue.cc:260] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.965966 10774 ts_tablet_manager.cc:1428] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:54.966920 10775 tablet_bootstrap.cc:492] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: No bootstrap required, opened a new log
I20260810 18:53:54.967012 10775 ts_tablet_manager.cc:1397] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260810 18:53:54.967440 10775 raft_consensus.cc:348] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.967541 10775 raft_consensus.cc:374] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:54.967571 10775 raft_consensus.cc:729] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f9f60c450b4d43843c81d24b9def4d, State: Initialized, Role: FOLLOWER
I20260810 18:53:54.967703 10775 consensus_queue.cc:260] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.967941 10775 ts_tablet_manager.cc:1428] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:54.969653 10812 ts_tablet_manager.cc:1918] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:54.969766 10812 log.cc:1196] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/db5c1d735ac049cc922a7afaa264e7cb
I20260810 18:53:54.970150 10812 ts_tablet_manager.cc:1939] T db5c1d735ac049cc922a7afaa264e7cb P 69f9f60c450b4d43843c81d24b9def4d: Deleting consensus metadata
I20260810 18:53:54.971385 10013 catalog_manager.cc:4841] TS 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199): tablet db5c1d735ac049cc922a7afaa264e7cb (table test-table [id=02847a24d01f4ba6b2290dedad40f10a]) successfully deleted
I20260810 18:53:54.971812 10014 catalog_manager.cc:4841] TS 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411): tablet db5c1d735ac049cc922a7afaa264e7cb (table test-table [id=02847a24d01f4ba6b2290dedad40f10a]) successfully deleted
I20260810 18:53:54.972559 10773 tablet_bootstrap.cc:492] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: No bootstrap required, opened a new log
I20260810 18:53:54.972666 10773 ts_tablet_manager.cc:1397] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20260810 18:53:54.973129 10773 raft_consensus.cc:348] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.973255 10773 raft_consensus.cc:374] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:54.973289 10773 raft_consensus.cc:729] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e233e06d7d453eb125f9a5031352af, State: Initialized, Role: FOLLOWER
I20260810 18:53:54.973416 10773 consensus_queue.cc:260] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:54.973666 10773 ts_tablet_manager.cc:1428] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260810 18:53:54.983999 10323 tablet.cc:2369] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:55.017088 10807 raft_consensus.cc:482] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:55.017273 10807 raft_consensus.cc:504] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.017740 10807 leader_election.cc:288] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053)
I20260810 18:53:55.018244 10256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a9c4789d08347d888b96c1ea4698c70" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2" is_pre_election: true
I20260810 18:53:55.018244 10470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a9c4789d08347d888b96c1ea4698c70" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e233e06d7d453eb125f9a5031352af" is_pre_election: true
I20260810 18:53:55.018448 10256 raft_consensus.cc:2454] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 0.
I20260810 18:53:55.018456 10470 raft_consensus.cc:2454] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 0.
I20260810 18:53:55.018755 10574 leader_election.cc:302] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69f9f60c450b4d43843c81d24b9def4d, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:55.018962 10807 raft_consensus.cc:2781] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:55.019032 10807 raft_consensus.cc:482] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:55.019089 10807 raft_consensus.cc:3037] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.020409 10807 raft_consensus.cc:504] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.020784 10807 leader_election.cc:288] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 election: Requested vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053)
I20260810 18:53:55.021186 10470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a9c4789d08347d888b96c1ea4698c70" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e233e06d7d453eb125f9a5031352af"
I20260810 18:53:55.021341 10470 raft_consensus.cc:3037] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.021364 10256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a9c4789d08347d888b96c1ea4698c70" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2"
I20260810 18:53:55.021495 10256 raft_consensus.cc:3037] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.022694 10470 raft_consensus.cc:2454] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 1.
I20260810 18:53:55.022722 10256 raft_consensus.cc:2454] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 1.
I20260810 18:53:55.022996 10574 leader_election.cc:302] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69f9f60c450b4d43843c81d24b9def4d, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:55.023175 10807 raft_consensus.cc:2781] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:55.023348 10807 raft_consensus.cc:686] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 1 LEADER]: Becoming Leader. State: Replica: 69f9f60c450b4d43843c81d24b9def4d, State: Running, Role: LEADER
I20260810 18:53:55.023488 10807 consensus_queue.cc:237] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.025249 10033 catalog_manager.cc:5495] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131). New cstate: current_term: 1 leader_uuid: "69f9f60c450b4d43843c81d24b9def4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } health_report { overall_health: HEALTHY } } }
Aug 10 18:53:55 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388033, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.190@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:55.070495 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.060512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.30.153.190:37787)
0810 18:53:55.060664 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.060669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.060856 (+ 187us) server_negotiation.cc:407] Connection header received
0810 18:53:55.061009 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.061012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.061060 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.061124 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:55.061934 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.062571 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.063703 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.064021 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.064141 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.066819 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:55.066823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:55.066828 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:55.066854 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:55.068525 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:55.068999 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:55.069003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:55.069005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:55.069133 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:55.069463 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:55.069467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:55.069468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:55.069697 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:55.069804 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.070164 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.070291 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:53:55.070494 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.060450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37787 (local address 127.0.0.1:36968)
0810 18:53:55.060754 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:55.060770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:55.060901 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:55.061187 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:55.061190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:55.061693 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:55.061816 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.061824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.062691 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.062695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.063535 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.063548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.064165 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.064169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.064252 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.064879 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:55.064894 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:55.066674 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:55.068646 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:55.068654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:55.068659 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:55.068890 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:55.069263 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:55.069265 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:55.069267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:55.069353 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:55.069856 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:55.069863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:55.069993 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 18:53:55.070086 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:53:55.070208 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":103,"threads_started":1}
I20260810 18:53:55.075625 10033 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:36968:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 18:53:55.076436 10033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:55.081974 10433 tablet_service.cc:1467] Processing CreateTablet for tablet dbf5695c37524c7e97eb117b1e8228ba (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab3df8862d66400f919239517b50d08e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:55.081974 10229 tablet_service.cc:1467] Processing CreateTablet for tablet dbf5695c37524c7e97eb117b1e8228ba (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab3df8862d66400f919239517b50d08e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:55.082499 10433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf5695c37524c7e97eb117b1e8228ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.082507 10229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf5695c37524c7e97eb117b1e8228ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.085274 10773 tablet_bootstrap.cc:492] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: Bootstrap starting.
I20260810 18:53:55.086128 10774 tablet_bootstrap.cc:492] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: Bootstrap starting.
I20260810 18:53:55.086199 10773 tablet_bootstrap.cc:654] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.086310 10644 tablet_service.cc:1467] Processing CreateTablet for tablet dbf5695c37524c7e97eb117b1e8228ba (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ab3df8862d66400f919239517b50d08e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 18:53:55.086761 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbf5695c37524c7e97eb117b1e8228ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.087131 10774 tablet_bootstrap.cc:654] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.088555 10774 tablet_bootstrap.cc:492] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: No bootstrap required, opened a new log
I20260810 18:53:55.088650 10774 ts_tablet_manager.cc:1397] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:55.089118 10774 raft_consensus.cc:348] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.089268 10774 raft_consensus.cc:374] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.089320 10774 raft_consensus.cc:729] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db36344207b4d1097f1be9df43e70a2, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.089494 10775 tablet_bootstrap.cc:492] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: Bootstrap starting.
I20260810 18:53:55.089453 10774 consensus_queue.cc:260] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.089835 10792 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.089979 10792 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:55.090261 10774 ts_tablet_manager.cc:1428] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:55.090948 10775 tablet_bootstrap.cc:654] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.091221 10773 tablet_bootstrap.cc:492] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: No bootstrap required, opened a new log
I20260810 18:53:55.091312 10773 ts_tablet_manager.cc:1397] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260810 18:53:55.091758 10773 raft_consensus.cc:348] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.091886 10773 raft_consensus.cc:374] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.091931 10773 raft_consensus.cc:729] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e233e06d7d453eb125f9a5031352af, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.092082 10773 consensus_queue.cc:260] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.092371 10773 ts_tablet_manager.cc:1428] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:55.092787 10775 tablet_bootstrap.cc:492] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: No bootstrap required, opened a new log
I20260810 18:53:55.092883 10775 ts_tablet_manager.cc:1397] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260810 18:53:55.092888 10789 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.093051 10789 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:55.093315 10775 raft_consensus.cc:348] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.093432 10775 raft_consensus.cc:374] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.093474 10775 raft_consensus.cc:729] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f9f60c450b4d43843c81d24b9def4d, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.093609 10775 consensus_queue.cc:260] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.093868 10775 ts_tablet_manager.cc:1428] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:53:55.093950 10807 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.094101 10807 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:55.122854 10789 raft_consensus.cc:482] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:55.123023 10789 raft_consensus.cc:504] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.124100 10789 leader_election.cc:288] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199)
I20260810 18:53:55.129588 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.124437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:36219 (local address 127.30.153.131:35199)
0810 18:53:55.124669 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.124674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.124806 (+ 132us) server_negotiation.cc:407] Connection header received
0810 18:53:55.124963 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.124965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.125010 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.125137 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:55.125481 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.126261 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.128286 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.128901 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.129029 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.129080 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.129383 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.129457 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260810 18:53:55.129588 10768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.124529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:44351 (local address 127.30.153.129:33411)
0810 18:53:55.124667 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.124672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.125091 (+ 419us) server_negotiation.cc:407] Connection header received
0810 18:53:55.125227 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.125230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.125283 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.125355 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:53:55.125657 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.126364 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.128286 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.128904 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.129029 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.129080 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.129383 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.129465 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:53:55.129674 10841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.124314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35199 (local address 127.30.153.130:36219)
0810 18:53:55.124700 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:55.124716 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:55.124831 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:55.125166 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:55.125172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:55.125192 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:55.125322 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.125327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.126379 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.126381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.128147 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.128153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.129034 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.129038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.129183 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.129249 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 18:53:55.129330 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 18:53:55.129456 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":320,"thread_start_us":129,"threads_started":1}
I20260810 18:53:55.129797 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.124314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:33411 (local address 127.30.153.130:44351)
0810 18:53:55.124991 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:55.125002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:55.125114 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:55.125399 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:55.125405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:55.125439 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:53:55.125536 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.125541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.126509 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.126512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.128137 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.128150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.129034 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.129038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.129234 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.129250 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 18:53:55.129330 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 18:53:55.129452 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":645,"thread_start_us":86,"threads_started":1}
I20260810 18:53:55.130374 10256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf5695c37524c7e97eb117b1e8228ba" candidate_uuid: "a9e233e06d7d453eb125f9a5031352af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2" is_pre_election: true
I20260810 18:53:55.130564 10256 raft_consensus.cc:2454] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9e233e06d7d453eb125f9a5031352af in term 0.
I20260810 18:53:55.130528 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf5695c37524c7e97eb117b1e8228ba" candidate_uuid: "a9e233e06d7d453eb125f9a5031352af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69f9f60c450b4d43843c81d24b9def4d" is_pre_election: true
I20260810 18:53:55.130789 10679 raft_consensus.cc:2454] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9e233e06d7d453eb125f9a5031352af in term 0.
I20260810 18:53:55.130970 10364 leader_election.cc:302] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db36344207b4d1097f1be9df43e70a2, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:55.131304 10789 raft_consensus.cc:2781] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:55.131388 10789 raft_consensus.cc:482] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:55.131444 10789 raft_consensus.cc:3037] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.132467 10789 raft_consensus.cc:504] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.132838 10789 leader_election.cc:288] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [CANDIDATE]: Term 1 election: Requested vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199)
I20260810 18:53:55.133175 10256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf5695c37524c7e97eb117b1e8228ba" candidate_uuid: "a9e233e06d7d453eb125f9a5031352af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2"
I20260810 18:53:55.133231 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbf5695c37524c7e97eb117b1e8228ba" candidate_uuid: "a9e233e06d7d453eb125f9a5031352af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69f9f60c450b4d43843c81d24b9def4d"
I20260810 18:53:55.133316 10256 raft_consensus.cc:3037] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.133361 10679 raft_consensus.cc:3037] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.134241 10256 raft_consensus.cc:2454] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9e233e06d7d453eb125f9a5031352af in term 1.
I20260810 18:53:55.134341 10679 raft_consensus.cc:2454] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9e233e06d7d453eb125f9a5031352af in term 1.
I20260810 18:53:55.134651 10364 leader_election.cc:302] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db36344207b4d1097f1be9df43e70a2, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:55.134816 10789 raft_consensus.cc:2781] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:55.135027 10789 raft_consensus.cc:686] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [term 1 LEADER]: Becoming Leader. State: Replica: a9e233e06d7d453eb125f9a5031352af, State: Running, Role: LEADER
I20260810 18:53:55.135283 10789 consensus_queue.cc:237] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.137629 10843 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: TxnStatusTablet state changed. Reason: New leader a9e233e06d7d453eb125f9a5031352af. Latest consensus state: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.137746 10843 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:55.138077 10846 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 18:53:55.138231 10846 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 18:53:55.138332 10033 catalog_manager.cc:5495] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af reported cstate change: term changed from 0 to 1, leader changed from <none> to a9e233e06d7d453eb125f9a5031352af (127.30.153.130). New cstate: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:55.157202 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.153500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33896 (local address 127.30.153.130:38053)
0810 18:53:55.153628 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.153635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.153652 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:53:55.153882 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.153884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.153940 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.154059 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:55.154329 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.154903 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.155762 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.155980 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.156095 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.156395 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.156799 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.156927 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.157003 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.157054 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:55.157200 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.153402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38053 (local address 127.0.0.1:33896)
0810 18:53:55.153593 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:55.153610 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:55.153726 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:55.154069 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:55.154072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:55.154080 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:55.154214 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.154221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.155017 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.155021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.155630 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.155638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.156089 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.156093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.156191 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.156234 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.156897 (+ 663us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.156914 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 18:53:55.156982 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:53:55.157065 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 18:53:55.160326 10256 raft_consensus.cc:1261] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Refusing update from remote peer a9e233e06d7d453eb125f9a5031352af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:55.160537 10679 raft_consensus.cc:1261] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Refusing update from remote peer a9e233e06d7d453eb125f9a5031352af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:55.160844 10843 consensus_queue.cc:1035] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:55.161082 10789 consensus_queue.cc:1035] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af [LEADER]: Connected to new peer: Peer: permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:55.162688 10792 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: TxnStatusTablet state changed. Reason: New leader a9e233e06d7d453eb125f9a5031352af. Latest consensus state: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.162828 10792 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:55.163229 10807 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: TxnStatusTablet state changed. Reason: New leader a9e233e06d7d453eb125f9a5031352af. Latest consensus state: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.163350 10807 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:55.164930 10843 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.165066 10843 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:55.165750 10807 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.165858 10807 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P 69f9f60c450b4d43843c81d24b9def4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:55.167737 10789 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.167853 10789 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P a9e233e06d7d453eb125f9a5031352af: This TxnStatusTablet replica's current role is: LEADER
I20260810 18:53:55.169104 10792 tablet_replica.cc:439] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } } }
I20260810 18:53:55.169221 10792 tablet_replica.cc:442] T dbf5695c37524c7e97eb117b1e8228ba P 2db36344207b4d1097f1be9df43e70a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 18:53:55.180557 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.177187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38053 (local address 127.30.153.131:47109)
0810 18:53:55.177371 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:55.177386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:55.177496 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:55.177778 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:55.177781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:55.177789 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:55.177898 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.177904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.178725 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.178729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.179351 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.179359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.179821 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.179825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.179912 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.179960 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.180289 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.180296 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:53:55.180358 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 18:53:55.180441 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 18:53:55.180557 10771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.177402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:47109 (local address 127.30.153.130:38053)
0810 18:53:55.177564 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.177569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.177584 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:53:55.177630 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.177633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.177682 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.177767 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:55.178014 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.178612 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.179482 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.179702 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.179809 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.180126 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.180193 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.180269 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.180367 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.180416 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:53:55.188104 10470 raft_consensus.cc:1261] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Refusing update from remote peer 69f9f60c450b4d43843c81d24b9def4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:55.188104 10256 raft_consensus.cc:1261] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Refusing update from remote peer 69f9f60c450b4d43843c81d24b9def4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:55.188540 10807 consensus_queue.cc:1035] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:55.188794 10794 consensus_queue.cc:1035] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:55.215310 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.211521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:46509 (local address 127.30.153.131:35199)
0810 18:53:55.211668 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.211673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.211948 (+ 275us) server_negotiation.cc:407] Connection header received
0810 18:53:55.212208 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.212212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.212260 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.212354 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:55.212642 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.213169 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.214102 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.214343 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.214492 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.214778 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.214892 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.214972 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.215112 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.215171 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:53:55.215332 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.211448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35199 (local address 127.30.153.130:46509)
0810 18:53:55.211861 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:55.211877 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:55.212088 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:55.212368 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:55.212371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:55.212379 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:55.212521 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.212527 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.213293 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.213296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.213968 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.213977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.214493 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.214496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.214597 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.214653 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.214993 (+ 340us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.215002 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:53:55.215078 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:53:55.215160 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":162,"threads_started":1}
I20260810 18:53:55.243902 10033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:55.244086 10033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:55.246286 10033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: Sending DeleteTablet for 3 replicas of tablet 8a9c4789d08347d888b96c1ea4698c70
I20260810 18:53:55.246992 10229 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9c4789d08347d888b96c1ea4698c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:55 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:37930
I20260810 18:53:55.246999 10644 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9c4789d08347d888b96c1ea4698c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:55 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:59050
I20260810 18:53:55.247179 10812 tablet_replica.cc:330] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: stopping tablet replica
I20260810 18:53:55.247180 10811 tablet_replica.cc:330] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: stopping tablet replica
I20260810 18:53:55.247295 10812 raft_consensus.cc:2229] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:55.247295 10811 raft_consensus.cc:2229] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:55.247411 10811 raft_consensus.cc:2258] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:55.247480 10812 raft_consensus.cc:2258] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:55.248082 10433 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a9c4789d08347d888b96c1ea4698c70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:55 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33888
I20260810 18:53:55.248258 10810 tablet_replica.cc:330] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: stopping tablet replica
I20260810 18:53:55.248358 10810 raft_consensus.cc:2229] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:55.248402 10812 ts_tablet_manager.cc:1905] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:55.248394 10811 ts_tablet_manager.cc:1905] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:55.248446 10810 raft_consensus.cc:2258] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:55.251381 10811 ts_tablet_manager.cc:1918] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:55.251381 10812 ts_tablet_manager.cc:1918] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:55.251459 10811 log.cc:1196] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/8a9c4789d08347d888b96c1ea4698c70
I20260810 18:53:55.251459 10812 log.cc:1196] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/8a9c4789d08347d888b96c1ea4698c70
I20260810 18:53:55.251734 10811 ts_tablet_manager.cc:1939] T 8a9c4789d08347d888b96c1ea4698c70 P 2db36344207b4d1097f1be9df43e70a2: Deleting consensus metadata
I20260810 18:53:55.251734 10812 ts_tablet_manager.cc:1939] T 8a9c4789d08347d888b96c1ea4698c70 P 69f9f60c450b4d43843c81d24b9def4d: Deleting consensus metadata
I20260810 18:53:55.252552 10013 catalog_manager.cc:4841] TS 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199): tablet 8a9c4789d08347d888b96c1ea4698c70 (table test-table [id=7e01d89f598d4be2a94012bc7e42512b]) successfully deleted
I20260810 18:53:55.252878 10014 catalog_manager.cc:4841] TS 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411): tablet 8a9c4789d08347d888b96c1ea4698c70 (table test-table [id=7e01d89f598d4be2a94012bc7e42512b]) successfully deleted
I20260810 18:53:55.255162 10810 ts_tablet_manager.cc:1905] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:55.258322 10810 ts_tablet_manager.cc:1918] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:55.258466 10810 log.cc:1196] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/8a9c4789d08347d888b96c1ea4698c70
I20260810 18:53:55.258846 10810 ts_tablet_manager.cc:1939] T 8a9c4789d08347d888b96c1ea4698c70 P a9e233e06d7d453eb125f9a5031352af: Deleting consensus metadata
I20260810 18:53:55.259660 10016 catalog_manager.cc:4841] TS a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053): tablet 8a9c4789d08347d888b96c1ea4698c70 (table test-table [id=7e01d89f598d4be2a94012bc7e42512b]) successfully deleted
Aug 10 18:53:55 dist-test-slave-68xk krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 18:53:55 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:53:55 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:53:55.282490 10105 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:53:55.275463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.30.153.190:37787)
0810 18:53:55.275708 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.275712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.276013 (+ 301us) server_negotiation.cc:407] Connection header received
0810 18:53:55.276245 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.276248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.276296 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.276391 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:55.277230 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.277769 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.278728 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.278958 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.279070 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.282299 (+ 3229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36978: BlockingRecv error: recv got EOF from 127.0.0.1:36978 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 18:53:55 dist-test-slave-68xk krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM
I20260810 18:53:55.293241 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.284297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.30.153.190:37787)
0810 18:53:55.284426 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.284429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.284640 (+ 211us) server_negotiation.cc:407] Connection header received
0810 18:53:55.284818 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.284821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.284866 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.284952 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:55.285596 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.286138 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.287012 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.287249 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.287362 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.289862 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:55.289865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:55.289867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:55.289893 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:55.291506 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:55.291958 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:55.291961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:55.291963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:55.292041 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:55.292331 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:55.292334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:55.292336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:55.292524 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:55.292672 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.292918 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.293043 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:53:55.296551 10033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:53:55.297027 10033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:55.302248 10433 tablet_service.cc:1467] Processing CreateTablet for tablet 69071cde24e243198814c85a1869c857 (DEFAULT_TABLE table=test-table [id=f2f84db0ae8446759aaae5347118d327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:55.302331 10644 tablet_service.cc:1467] Processing CreateTablet for tablet 69071cde24e243198814c85a1869c857 (DEFAULT_TABLE table=test-table [id=f2f84db0ae8446759aaae5347118d327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:55.302728 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69071cde24e243198814c85a1869c857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.302726 10229 tablet_service.cc:1467] Processing CreateTablet for tablet 69071cde24e243198814c85a1869c857 (DEFAULT_TABLE table=test-table [id=f2f84db0ae8446759aaae5347118d327]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:55.303052 10229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69071cde24e243198814c85a1869c857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.304780 10775 tablet_bootstrap.cc:492] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: Bootstrap starting.
I20260810 18:53:55.305595 10775 tablet_bootstrap.cc:654] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.305727 10774 tablet_bootstrap.cc:492] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: Bootstrap starting.
I20260810 18:53:55.306264 10433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69071cde24e243198814c85a1869c857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.306737 10774 tablet_bootstrap.cc:654] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.307840 10774 tablet_bootstrap.cc:492] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: No bootstrap required, opened a new log
I20260810 18:53:55.307952 10774 ts_tablet_manager.cc:1397] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:55.308430 10774 raft_consensus.cc:348] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.308549 10774 raft_consensus.cc:374] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.308615 10774 raft_consensus.cc:729] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db36344207b4d1097f1be9df43e70a2, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.308779 10774 consensus_queue.cc:260] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.309074 10774 ts_tablet_manager.cc:1428] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:55.309305 10773 tablet_bootstrap.cc:492] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: Bootstrap starting.
I20260810 18:53:55.309854 10775 tablet_bootstrap.cc:492] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: No bootstrap required, opened a new log
I20260810 18:53:55.309933 10775 ts_tablet_manager.cc:1397] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260810 18:53:55.310309 10773 tablet_bootstrap.cc:654] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.310367 10775 raft_consensus.cc:348] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.310516 10775 raft_consensus.cc:374] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.310555 10775 raft_consensus.cc:729] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f9f60c450b4d43843c81d24b9def4d, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.310683 10775 consensus_queue.cc:260] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.310942 10775 ts_tablet_manager.cc:1428] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:55.312613 10773 tablet_bootstrap.cc:492] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: No bootstrap required, opened a new log
I20260810 18:53:55.312705 10773 ts_tablet_manager.cc:1397] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:55.313537 10773 raft_consensus.cc:348] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.313659 10773 raft_consensus.cc:374] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.313702 10773 raft_consensus.cc:729] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e233e06d7d453eb125f9a5031352af, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.313846 10773 consensus_queue.cc:260] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.314159 10773 ts_tablet_manager.cc:1428] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260810 18:53:55.448653 10739 tablet.cc:2369] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:53:55.467664 10536 tablet.cc:2369] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:53:55.502830 10790 raft_consensus.cc:482] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:55.502983 10790 raft_consensus.cc:504] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.503437 10790 leader_election.cc:288] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053)
I20260810 18:53:55.503939 10470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69071cde24e243198814c85a1869c857" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e233e06d7d453eb125f9a5031352af" is_pre_election: true
I20260810 18:53:55.503955 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69071cde24e243198814c85a1869c857" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2" is_pre_election: true
I20260810 18:53:55.504136 10470 raft_consensus.cc:2454] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 0.
I20260810 18:53:55.504150 10255 raft_consensus.cc:2454] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 0.
I20260810 18:53:55.504530 10574 leader_election.cc:302] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69f9f60c450b4d43843c81d24b9def4d, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:55.504758 10790 raft_consensus.cc:2781] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:55.504832 10790 raft_consensus.cc:482] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:55.504914 10790 raft_consensus.cc:3037] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.506048 10790 raft_consensus.cc:504] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.506510 10790 leader_election.cc:288] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 election: Requested vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053)
I20260810 18:53:55.506922 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69071cde24e243198814c85a1869c857" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2"
I20260810 18:53:55.506922 10470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69071cde24e243198814c85a1869c857" candidate_uuid: "69f9f60c450b4d43843c81d24b9def4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9e233e06d7d453eb125f9a5031352af"
I20260810 18:53:55.507059 10255 raft_consensus.cc:3037] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.507059 10470 raft_consensus.cc:3037] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.508368 10470 raft_consensus.cc:2454] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 1.
I20260810 18:53:55.508415 10255 raft_consensus.cc:2454] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69f9f60c450b4d43843c81d24b9def4d in term 1.
I20260810 18:53:55.508711 10574 leader_election.cc:302] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69f9f60c450b4d43843c81d24b9def4d, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:55.508919 10790 raft_consensus.cc:2781] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:55.509023 10790 raft_consensus.cc:686] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 1 LEADER]: Becoming Leader. State: Replica: 69f9f60c450b4d43843c81d24b9def4d, State: Running, Role: LEADER
I20260810 18:53:55.509212 10790 consensus_queue.cc:237] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.511078 10029 catalog_manager.cc:5495] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131). New cstate: current_term: 1 leader_uuid: "69f9f60c450b4d43843c81d24b9def4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } health_report { overall_health: HEALTHY } } }
I20260810 18:53:55.546289 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.543335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59072 (local address 127.30.153.131:35199)
0810 18:53:55.543485 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.543489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.543503 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:53:55.543630 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.543633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.543684 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.543749 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:55.543998 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.544486 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.545315 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.545521 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.545596 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.545847 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.545916 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.545972 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.546062 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.546109 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:53:55.548382 10255 raft_consensus.cc:1261] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Refusing update from remote peer 69f9f60c450b4d43843c81d24b9def4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:55.548380 10470 raft_consensus.cc:1261] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Refusing update from remote peer 69f9f60c450b4d43843c81d24b9def4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:55.548851 10790 consensus_queue.cc:1035] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:55.549062 10794 consensus_queue.cc:1035] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:55.556079 10031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:55.556241 10031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:55.558403 10031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: Sending DeleteTablet for 3 replicas of tablet 69071cde24e243198814c85a1869c857
I20260810 18:53:55.558975 10229 tablet_service.cc:1514] Processing DeleteTablet for tablet 69071cde24e243198814c85a1869c857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:55 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:37930
I20260810 18:53:55.559203 10811 tablet_replica.cc:330] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: stopping tablet replica
I20260810 18:53:55.559340 10811 raft_consensus.cc:2229] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:55.559499 10811 raft_consensus.cc:2258] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:55.559520 10644 tablet_service.cc:1514] Processing DeleteTablet for tablet 69071cde24e243198814c85a1869c857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:55 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:59050
I20260810 18:53:55.559545 10433 tablet_service.cc:1514] Processing DeleteTablet for tablet 69071cde24e243198814c85a1869c857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:55 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33888
I20260810 18:53:55.559701 10812 tablet_replica.cc:330] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: stopping tablet replica
I20260810 18:53:55.559823 10812 raft_consensus.cc:2229] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:55.559930 10031 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 18:53:55.560045 10812 raft_consensus.cc:2258] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:55.560217 10810 tablet_replica.cc:330] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: stopping tablet replica
W20260810 18:53:55.560333 10031 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:53:55.560339 10810 raft_consensus.cc:2229] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:55.560485 10810 raft_consensus.cc:2258] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:55.561234 10812 ts_tablet_manager.cc:1905] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:55.561304 10810 ts_tablet_manager.cc:1905] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:55.561540 10811 ts_tablet_manager.cc:1905] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:55.563493 10810 ts_tablet_manager.cc:1918] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:55.563597 10810 log.cc:1196] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/69071cde24e243198814c85a1869c857
I20260810 18:53:55.563807 10812 ts_tablet_manager.cc:1918] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:55.563891 10812 log.cc:1196] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/69071cde24e243198814c85a1869c857
I20260810 18:53:55.563920 10810 ts_tablet_manager.cc:1939] T 69071cde24e243198814c85a1869c857 P a9e233e06d7d453eb125f9a5031352af: Deleting consensus metadata
I20260810 18:53:55.563987 10811 ts_tablet_manager.cc:1918] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:53:55.564065 10811 log.cc:1196] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/69071cde24e243198814c85a1869c857
I20260810 18:53:55.564179 10812 ts_tablet_manager.cc:1939] T 69071cde24e243198814c85a1869c857 P 69f9f60c450b4d43843c81d24b9def4d: Deleting consensus metadata
I20260810 18:53:55.564366 10811 ts_tablet_manager.cc:1939] T 69071cde24e243198814c85a1869c857 P 2db36344207b4d1097f1be9df43e70a2: Deleting consensus metadata
I20260810 18:53:55.564754 10016 catalog_manager.cc:4841] TS a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053): tablet 69071cde24e243198814c85a1869c857 (table test-table [id=f2f84db0ae8446759aaae5347118d327]) successfully deleted
I20260810 18:53:55.564994 10013 catalog_manager.cc:4841] TS 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199): tablet 69071cde24e243198814c85a1869c857 (table test-table [id=f2f84db0ae8446759aaae5347118d327]) successfully deleted
I20260810 18:53:55.565230 10014 catalog_manager.cc:4841] TS 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411): tablet 69071cde24e243198814c85a1869c857 (table test-table [id=f2f84db0ae8446759aaae5347118d327]) successfully deleted
I20260810 18:53:55.568228 10229 tablet_service.cc:1467] Processing CreateTablet for tablet 1af7aebc41ed4c109c0eae04d43ea3cc (DEFAULT_TABLE table=test-table [id=3f92a9c75c074b8182dd82924e3fd7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:55.568437 10644 tablet_service.cc:1467] Processing CreateTablet for tablet 1af7aebc41ed4c109c0eae04d43ea3cc (DEFAULT_TABLE table=test-table [id=3f92a9c75c074b8182dd82924e3fd7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:55.568507 10229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1af7aebc41ed4c109c0eae04d43ea3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.568720 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1af7aebc41ed4c109c0eae04d43ea3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.570534 10774 tablet_bootstrap.cc:492] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: Bootstrap starting.
I20260810 18:53:55.570782 10433 tablet_service.cc:1467] Processing CreateTablet for tablet 1af7aebc41ed4c109c0eae04d43ea3cc (DEFAULT_TABLE table=test-table [id=3f92a9c75c074b8182dd82924e3fd7b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:53:55.571056 10433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1af7aebc41ed4c109c0eae04d43ea3cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:55.571111 10775 tablet_bootstrap.cc:492] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: Bootstrap starting.
I20260810 18:53:55.571593 10774 tablet_bootstrap.cc:654] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.571990 10775 tablet_bootstrap.cc:654] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.573155 10774 tablet_bootstrap.cc:492] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: No bootstrap required, opened a new log
I20260810 18:53:55.573181 10775 tablet_bootstrap.cc:492] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: No bootstrap required, opened a new log
I20260810 18:53:55.573266 10774 ts_tablet_manager.cc:1397] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:55.573266 10775 ts_tablet_manager.cc:1397] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:55.573321 10773 tablet_bootstrap.cc:492] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: Bootstrap starting.
I20260810 18:53:55.573668 10774 raft_consensus.cc:348] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.573668 10775 raft_consensus.cc:348] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.573778 10775 raft_consensus.cc:374] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.573778 10774 raft_consensus.cc:374] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.573813 10774 raft_consensus.cc:729] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db36344207b4d1097f1be9df43e70a2, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.573813 10775 raft_consensus.cc:729] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f9f60c450b4d43843c81d24b9def4d, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.573949 10775 consensus_queue.cc:260] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.573961 10774 consensus_queue.cc:260] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.574126 10773 tablet_bootstrap.cc:654] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:55.574222 10775 ts_tablet_manager.cc:1428] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:53:55.575281 10773 tablet_bootstrap.cc:492] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: No bootstrap required, opened a new log
I20260810 18:53:55.575402 10773 ts_tablet_manager.cc:1397] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260810 18:53:55.575796 10773 raft_consensus.cc:348] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.575924 10773 raft_consensus.cc:374] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:55.575955 10774 ts_tablet_manager.cc:1428] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260810 18:53:55.575968 10773 raft_consensus.cc:729] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9e233e06d7d453eb125f9a5031352af, State: Initialized, Role: FOLLOWER
I20260810 18:53:55.576107 10773 consensus_queue.cc:260] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.576382 10773 ts_tablet_manager.cc:1428] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260810 18:53:55.916788 10853 raft_consensus.cc:482] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:53:55.916962 10853 raft_consensus.cc:504] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.917574 10853 leader_election.cc:288] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199)
I20260810 18:53:55.918109 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1af7aebc41ed4c109c0eae04d43ea3cc" candidate_uuid: "a9e233e06d7d453eb125f9a5031352af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2" is_pre_election: true
I20260810 18:53:55.918217 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1af7aebc41ed4c109c0eae04d43ea3cc" candidate_uuid: "a9e233e06d7d453eb125f9a5031352af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69f9f60c450b4d43843c81d24b9def4d" is_pre_election: true
I20260810 18:53:55.918304 10255 raft_consensus.cc:2454] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9e233e06d7d453eb125f9a5031352af in term 0.
I20260810 18:53:55.918452 10679 raft_consensus.cc:2454] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9e233e06d7d453eb125f9a5031352af in term 0.
I20260810 18:53:55.918756 10364 leader_election.cc:302] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db36344207b4d1097f1be9df43e70a2, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:55.918958 10853 raft_consensus.cc:2781] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:53:55.919083 10853 raft_consensus.cc:482] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:53:55.919209 10853 raft_consensus.cc:3037] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.920411 10853 raft_consensus.cc:504] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.920816 10853 leader_election.cc:288] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [CANDIDATE]: Term 1 election: Requested vote from peers 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411), 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199)
I20260810 18:53:55.921146 10255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1af7aebc41ed4c109c0eae04d43ea3cc" candidate_uuid: "a9e233e06d7d453eb125f9a5031352af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2db36344207b4d1097f1be9df43e70a2"
I20260810 18:53:55.921227 10679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1af7aebc41ed4c109c0eae04d43ea3cc" candidate_uuid: "a9e233e06d7d453eb125f9a5031352af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69f9f60c450b4d43843c81d24b9def4d"
I20260810 18:53:55.921272 10255 raft_consensus.cc:3037] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.921391 10679 raft_consensus.cc:3037] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:55.922263 10255 raft_consensus.cc:2454] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9e233e06d7d453eb125f9a5031352af in term 1.
I20260810 18:53:55.922406 10679 raft_consensus.cc:2454] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9e233e06d7d453eb125f9a5031352af in term 1.
I20260810 18:53:55.922606 10364 leader_election.cc:302] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2db36344207b4d1097f1be9df43e70a2, a9e233e06d7d453eb125f9a5031352af; no voters:
I20260810 18:53:55.922868 10853 raft_consensus.cc:2781] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:55.923024 10853 raft_consensus.cc:686] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 1 LEADER]: Becoming Leader. State: Replica: a9e233e06d7d453eb125f9a5031352af, State: Running, Role: LEADER
I20260810 18:53:55.923187 10853 consensus_queue.cc:237] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } }
I20260810 18:53:55.924811 10031 catalog_manager.cc:5495] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af reported cstate change: term changed from 0 to 1, leader changed from <none> to a9e233e06d7d453eb125f9a5031352af (127.30.153.130). New cstate: current_term: 1 leader_uuid: "a9e233e06d7d453eb125f9a5031352af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9e233e06d7d453eb125f9a5031352af" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 } health_report { overall_health: UNKNOWN } } }
I20260810 18:53:55.949926 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.946817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33908 (local address 127.30.153.130:38053)
0810 18:53:55.947065 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.947069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.947083 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:53:55.947125 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.947128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.947175 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.947253 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:55.947467 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.948029 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.948816 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.949062 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.949163 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.949407 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.949478 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.949583 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.949699 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.949756 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
I20260810 18:53:55.959713 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.956338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:41975 (local address 127.30.153.130:38053)
0810 18:53:55.956503 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:55.956508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:55.956645 (+ 137us) server_negotiation.cc:407] Connection header received
0810 18:53:55.956788 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:55.956792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:55.956843 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:55.956921 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:53:55.957153 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.957715 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.958612 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.958906 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.959017 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.959266 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.959344 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.959426 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:55.959518 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 18:53:55.959559 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:53:55.959713 10897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:55.956227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38053 (local address 127.30.153.130:41975)
0810 18:53:55.956545 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:55.956565 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:55.956684 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:55.956940 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:55.956942 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:55.956949 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 18:53:55.957040 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.957045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.957829 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.957835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.958482 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:55.958492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:55.959019 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:55.959023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:55.959119 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:55.959148 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 18:53:55.959430 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 18:53:55.959438 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:53:55.959500 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 18:53:55.959574 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260810 18:53:55.965441 10255 raft_consensus.cc:1261] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Refusing update from remote peer a9e233e06d7d453eb125f9a5031352af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:55.965478 10679 raft_consensus.cc:1261] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Refusing update from remote peer a9e233e06d7d453eb125f9a5031352af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:53:55.965934 10853 consensus_queue.cc:1035] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [LEADER]: Connected to new peer: Peer: permanent_uuid: "2db36344207b4d1097f1be9df43e70a2" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:55.966095 10898 consensus_queue.cc:1035] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [LEADER]: Connected to new peer: Peer: permanent_uuid: "69f9f60c450b4d43843c81d24b9def4d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:53:56.002769 10031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:56.002933 10031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:53:56.005499 10031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e77163c28e84b35932172e0a58784e9: Sending DeleteTablet for 3 replicas of tablet 1af7aebc41ed4c109c0eae04d43ea3cc
I20260810 18:53:56.006177 10229 tablet_service.cc:1514] Processing DeleteTablet for tablet 1af7aebc41ed4c109c0eae04d43ea3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:56 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:37930
I20260810 18:53:56.006338 10644 tablet_service.cc:1514] Processing DeleteTablet for tablet 1af7aebc41ed4c109c0eae04d43ea3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:56 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:59050
I20260810 18:53:56.006412 10811 tablet_replica.cc:330] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: stopping tablet replica
I20260810 18:53:56.006475 10431 tablet_service.cc:1514] Processing DeleteTablet for tablet 1af7aebc41ed4c109c0eae04d43ea3cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:53:56 UTC) from {username='oryx', principal='oryx/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:33888
I20260810 18:53:56.006587 10811 raft_consensus.cc:2229] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:56.006672 10810 tablet_replica.cc:330] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: stopping tablet replica
I20260810 18:53:56.006683 10812 tablet_replica.cc:330] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: stopping tablet replica
I20260810 18:53:56.006726 10811 raft_consensus.cc:2258] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:56.006798 10810 raft_consensus.cc:2229] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:53:56.006806 10812 raft_consensus.cc:2229] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:53:56.006929 10812 raft_consensus.cc:2258] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:56.007103 10810 raft_consensus.cc:2258] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:53:56.007804 10811 ts_tablet_manager.cc:1905] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:56.007941 10812 ts_tablet_manager.cc:1905] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:56.010178 10812 ts_tablet_manager.cc:1918] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:56.010272 10812 log.cc:1196] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/1af7aebc41ed4c109c0eae04d43ea3cc
I20260810 18:53:56.010576 10811 ts_tablet_manager.cc:1918] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:56.010612 10812 ts_tablet_manager.cc:1939] T 1af7aebc41ed4c109c0eae04d43ea3cc P 69f9f60c450b4d43843c81d24b9def4d: Deleting consensus metadata
I20260810 18:53:56.010660 10811 log.cc:1196] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/1af7aebc41ed4c109c0eae04d43ea3cc
I20260810 18:53:56.010950 10811 ts_tablet_manager.cc:1939] T 1af7aebc41ed4c109c0eae04d43ea3cc P 2db36344207b4d1097f1be9df43e70a2: Deleting consensus metadata
I20260810 18:53:56.011194 10810 ts_tablet_manager.cc:1905] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:53:56.011412 10013 catalog_manager.cc:4841] TS 69f9f60c450b4d43843c81d24b9def4d (127.30.153.131:35199): tablet 1af7aebc41ed4c109c0eae04d43ea3cc (table test-table [id=3f92a9c75c074b8182dd82924e3fd7b1]) successfully deleted
I20260810 18:53:56.013321 10810 ts_tablet_manager.cc:1918] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 18:53:56.013381 10810 log.cc:1196] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/1af7aebc41ed4c109c0eae04d43ea3cc
I20260810 18:53:56.013592 10810 ts_tablet_manager.cc:1939] T 1af7aebc41ed4c109c0eae04d43ea3cc P a9e233e06d7d453eb125f9a5031352af: Deleting consensus metadata
I20260810 18:53:56.013906 10014 catalog_manager.cc:4841] TS 2db36344207b4d1097f1be9df43e70a2 (127.30.153.129:33411): tablet 1af7aebc41ed4c109c0eae04d43ea3cc (table test-table [id=3f92a9c75c074b8182dd82924e3fd7b1]) successfully deleted
I20260810 18:53:56.014145 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 10136
I20260810 18:53:56.023591 10016 catalog_manager.cc:4841] TS a9e233e06d7d453eb125f9a5031352af (127.30.153.130:38053): tablet 1af7aebc41ed4c109c0eae04d43ea3cc (table test-table [id=3f92a9c75c074b8182dd82924e3fd7b1]) successfully deleted
I20260810 18:53:56.030098 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 10335
I20260810 18:53:56.043862 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 10548
I20260810 18:53:56.057617 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 9988
2026-08-10T18:53:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3754 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:56 dist-test-slave-68xk krb5kdc[10908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:56 dist-test-slave-68xk krb5kdc[10908](info): set up 2 sockets
Aug 10 18:53:56 dist-test-slave-68xk krb5kdc[10908](info): commencing operation
krb5kdc: starting...
W20260810 18:53:57.355860 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:57 dist-test-slave-68xk krb5kdc[10908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:57.495352 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:41617
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:41617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:57.614029 10925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:57.614367 10925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:57.614506 10925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:57.618544 10925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:57.618645 10925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:57.618683 10925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:57.618721 10925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:57.618758 10925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:57.624323 10925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:41617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:41617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:57.625471 10925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:57.626598 10925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:57.635393 10941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:57.635389 10937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:57.635489 10934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:57.635455 10925 server_base.cc:1034] running on GCE node
I20260810 18:53:57.636439 10925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:57.637567 10925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:57.638764 10925 hybrid_clock.cc:648] HybridClock initialized: now 1786388037638734 us; error 45 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T18:53:57Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1591 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:53:57 dist-test-slave-68xk krb5kdc[10946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:53:57 dist-test-slave-68xk krb5kdc[10946](info): set up 2 sockets
Aug 10 18:53:57 dist-test-slave-68xk krb5kdc[10946](info): commencing operation
krb5kdc: starting...
W20260810 18:53:58.963038 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:53:59 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:53:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:53:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:53:59.098104 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:46099
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:46099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:59.215271 10966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:59.215596 10966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:59.215696 10966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:59.219801 10966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:53:59.219887 10966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:59.219925 10966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:59.219985 10966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:53:59.220022 10966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:53:59.225863 10966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:46099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:46099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:59.227113 10966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:59.228196 10966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:59.237564 10972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:59.237671 10974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:59.237663 10966 server_base.cc:1034] running on GCE node
W20260810 18:53:59.237690 10977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:59.238651 10966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:59.239861 10966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:59.241066 10966 hybrid_clock.cc:648] HybridClock initialized: now 1786388039241048 us; error 30 us; skew 500 ppm
Aug 10 18:53:59 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:59.244329 10966 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:53:59.245340 10966 webserver.cc:460] Webserver started at http://127.30.153.190:41297/ using document root <none> and password file <none>
I20260810 18:53:59.245955 10966 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:59.246029 10966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:59.246332 10966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:59.248610 10966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "7d9f16edc4be404e989d3c37262bceb0"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "ccac1c6ecf264a5ce22d1e8d613cc919"
server_key_iv: "4cdbfb1e61e554d01676273dd4b029d7"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.249256 10966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "7d9f16edc4be404e989d3c37262bceb0"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "ccac1c6ecf264a5ce22d1e8d613cc919"
server_key_iv: "4cdbfb1e61e554d01676273dd4b029d7"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.253432 10966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:59.256258 10985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:59.257614 10966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:59.257782 10966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "7d9f16edc4be404e989d3c37262bceb0"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "ccac1c6ecf264a5ce22d1e8d613cc919"
server_key_iv: "4cdbfb1e61e554d01676273dd4b029d7"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.257906 10966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:59.274657 10966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:59.277714 10966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:59.277935 10966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:59.285179 10966 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:46099
I20260810 18:53:59.285195 11057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:46099 every 8 connection(s)
I20260810 18:53:59.286154 10966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:53:59.289131 11058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:53:59.295109 11058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Bootstrap starting.
I20260810 18:53:59.296322 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 10966
I20260810 18:53:59.296486 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:53:59.296821 31334 external_mini_cluster.cc:1428] Setting key e6863644e50c6076c80734a74b16e333
I20260810 18:53:59.298152 11058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Neither blocks nor log segments found. Creating new log.
I20260810 18:53:59.299037 11058 log.cc:824] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:53:59.301028 11058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: No bootstrap required, opened a new log
I20260810 18:53:59.304973 11058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } }
I20260810 18:53:59.305266 11058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:53:59.305352 11058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d9f16edc4be404e989d3c37262bceb0, State: Initialized, Role: FOLLOWER
I20260810 18:53:59.305933 11058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } }
Aug 10 18:53:59 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:59.306113 11058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:53:59.306193 11058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:53:59.306303 11058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:53:59.307520 11058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } }
I20260810 18:53:59.307945 11058 leader_election.cc:302] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d9f16edc4be404e989d3c37262bceb0; no voters:
I20260810 18:53:59.308358 11058 leader_election.cc:288] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:53:59.308498 11066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:53:59.308765 11066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 1 LEADER]: Becoming Leader. State: Replica: 7d9f16edc4be404e989d3c37262bceb0, State: Running, Role: LEADER
I20260810 18:53:59.309172 11066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } }
I20260810 18:53:59.309826 11058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:53:59.310940 11070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d9f16edc4be404e989d3c37262bceb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } } }
I20260810 18:53:59.311072 11070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:59.311326 11070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d9f16edc4be404e989d3c37262bceb0. Latest consensus state: current_term: 1 leader_uuid: "7d9f16edc4be404e989d3c37262bceb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } } }
I20260810 18:53:59.311408 11070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:53:59.313161 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:59.298056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.30.153.190:46099)
0810 18:53:59.298610 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:59.298636 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:59.298671 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:53:59.299425 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:59.299446 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:59.299791 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:59.300140 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:59.301012 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.302042 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.302974 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.303462 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.303642 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:59.306847 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:59.306852 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:59.306866 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:59.306895 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:59.309134 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.309654 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.309661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.309667 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.309817 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.310172 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.310175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.310177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.310784 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:59.311817 (+ 1033us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:59.311970 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 18:53:59.312316 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":231,"threads_started":1}
W20260810 18:53:59.315138 11089 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:53:59.315258 11089 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:53:59.315326 11080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:53:59.316200 11080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:53:59.322016 11080 catalog_manager.cc:1347] Generated new cluster ID: df003bf528c14bf699f90249ac0dae3e
I20260810 18:53:59.322091 11080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:53:59.346776 11080 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:53:59.347851 11080 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:53:59.356827 11080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Generated new TSK 0
I20260810 18:53:59.357684 11080 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:53:59.401546 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:46099
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:59.514536 11097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:59.514825 11097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:59.514900 11097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:59.518860 11097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:59.518934 11097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:59.519021 11097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:53:59.524080 11097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:59.525111 11097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:59.526109 11097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:53:59.535753 11097 server_base.cc:1034] running on GCE node
W20260810 18:53:59.535611 11104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:59.535606 11107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:59.535610 11105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:59.536597 11097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:59.537180 11097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:59.538342 11097 hybrid_clock.cc:648] HybridClock initialized: now 1786388039538324 us; error 33 us; skew 500 ppm
Aug 10 18:53:59 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:59.541253 11097 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:53:59.542050 11097 webserver.cc:460] Webserver started at http://127.30.153.129:35277/ using document root <none> and password file <none>
I20260810 18:53:59.542639 11097 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:59.542708 11097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:59.542872 11097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:59.544835 11097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "12093e00756e44b5bfe854578147396d"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "0c20bcbbfcf927f42e1fc7ef1cb252a8"
server_key_iv: "98c16bdb8c3eaa09802fb147e7b7dad2"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.545360 11097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "12093e00756e44b5bfe854578147396d"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "0c20bcbbfcf927f42e1fc7ef1cb252a8"
server_key_iv: "98c16bdb8c3eaa09802fb147e7b7dad2"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.549594 11097 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:59.552361 11117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:59.553558 11097 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:53:59.553699 11097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "12093e00756e44b5bfe854578147396d"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "0c20bcbbfcf927f42e1fc7ef1cb252a8"
server_key_iv: "98c16bdb8c3eaa09802fb147e7b7dad2"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.553810 11097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:59.565809 11097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:59.569098 11097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:59.569300 11097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:59.569888 11097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:59.570756 11097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:59.570803 11097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:59.570869 11097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:59.570902 11097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:59.580099 11097 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:33569
I20260810 18:53:59.580147 11274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:33569 every 8 connection(s)
I20260810 18:53:59.580986 11097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:53:59.589035 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 11097
I20260810 18:53:59.589221 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:53:59.589658 31334 external_mini_cluster.cc:1428] Setting key 260a9691d6d30dde0435edc536987882
Aug 10 18:53:59 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:53:59.596024 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:59.582837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:51415 (local address 127.30.153.190:46099)
0810 18:53:59.582990 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:59.582994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:59.583814 (+ 820us) server_negotiation.cc:407] Connection header received
0810 18:53:59.585109 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:59.585113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:59.585222 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:59.585320 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:59.586510 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.587053 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.588224 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.588479 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.588597 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:59.591717 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:59.591722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:59.591725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:59.591758 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:59.593521 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.594265 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.594268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.594269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.594348 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.594805 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.594807 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.594808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.594988 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:59.595069 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:59.595701 (+ 632us) server_negotiation.cc:299] Negotiation successful
0810 18:53:59.595842 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:53:59.596897 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:59.583189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:51415)
0810 18:53:59.583657 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:59.583699 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:59.584878 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:59.585502 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:59.585509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:59.585864 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:59.586326 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:59.586339 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.587195 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.587199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:59.588091 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:59.588100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.588650 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.588655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:59.588895 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:59.589638 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:59.589660 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:59.591529 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:59.593693 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:59.593702 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:59.593723 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:59.594140 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:59.594473 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:59.594478 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:59.594481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:59.594700 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:59.595105 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:59.595115 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:59.595425 (+ 310us) client_negotiation.cc:769] Sending connection context
0810 18:53:59.595634 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 18:53:59.595981 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":118,"threads_started":1}
I20260810 18:53:59.598512 11275 heartbeater.cc:347] Connected to a master server at 127.30.153.190:46099
I20260810 18:53:59.598803 11275 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:59.599361 11275 heartbeater.cc:511] Master 127.30.153.190:46099 requested a full tablet report, sending...
I20260810 18:53:59.601305 11007 ts_manager.cc:194] Registered new tserver with Master: 12093e00756e44b5bfe854578147396d (127.30.153.129:33569)
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:53:59.602762 11007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:51415
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:53:59.621815 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:46099
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:59.737246 11283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:59.737564 11283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:59.737637 11283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:59.741619 11283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:59.741716 11283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:59.741811 11283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:53:59.747038 11283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:59.748132 11283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:59.749190 11283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:59.759228 11291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:59.759294 11292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:59.759357 11283 server_base.cc:1034] running on GCE node
W20260810 18:53:59.759302 11294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:59.759967 11283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:59.760711 11283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:59.761906 11283 hybrid_clock.cc:648] HybridClock initialized: now 1786388039761856 us; error 60 us; skew 500 ppm
Aug 10 18:53:59 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:59.765014 11283 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:53:59.765996 11283 webserver.cc:460] Webserver started at http://127.30.153.130:35699/ using document root <none> and password file <none>
I20260810 18:53:59.766650 11283 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:59.766721 11283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:59.766899 11283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:53:59.769104 11283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "41fefa851c1c47d2b96604fe67d23f1a"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "d21c1d53cfbd091310d0251755e97648"
server_key_iv: "d5bffbfda68a0e0d335ef1ee32113432"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.769660 11283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "41fefa851c1c47d2b96604fe67d23f1a"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "d21c1d53cfbd091310d0251755e97648"
server_key_iv: "d5bffbfda68a0e0d335ef1ee32113432"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.773752 11283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:53:59.776655 11301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:59.778092 11283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:53:59.778242 11283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "41fefa851c1c47d2b96604fe67d23f1a"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "d21c1d53cfbd091310d0251755e97648"
server_key_iv: "d5bffbfda68a0e0d335ef1ee32113432"
server_key_version: "encryptionkey@0"
I20260810 18:53:59.778362 11283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:53:59.790581 11283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:53:59.794060 11283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:53:59.794317 11283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:53:59.795069 11283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:53:59.796037 11283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:53:59.796135 11283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:59.796211 11283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:53:59.796268 11283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:53:59.807082 11283 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:36365
I20260810 18:53:59.807106 11487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:36365 every 8 connection(s)
I20260810 18:53:59.808171 11283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:53:59.809373 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 11283
I20260810 18:53:59.809532 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:53:59.809940 31334 external_mini_cluster.cc:1428] Setting key f8363779e59723393afa0f3d7fc35c62
Aug 10 18:53:59 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:53:59.825731 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:59.810597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:39433 (local address 127.30.153.190:46099)
0810 18:53:59.810751 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:53:59.810757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:53:59.812688 (+ 1931us) server_negotiation.cc:407] Connection header received
0810 18:53:59.814309 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:53:59.814316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:53:59.814388 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:53:59.814529 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:53:59.815839 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.816467 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.817762 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.818013 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.818145 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:59.821657 (+ 3512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:53:59.821661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:53:59.821663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:53:59.821689 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:53:59.823469 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.823997 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.824000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.824001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.824080 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:53:59.824522 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:53:59.824524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:53:59.824525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:53:59.824710 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:53:59.824790 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:53:59.825491 (+ 701us) server_negotiation.cc:299] Negotiation successful
0810 18:53:59.825599 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
I20260810 18:53:59.827011 11493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:53:59.811213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:39433)
0810 18:53:59.812502 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0810 18:53:59.812572 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0810 18:53:59.813969 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:53:59.814725 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:53:59.814733 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:53:59.815155 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:53:59.815668 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:59.815682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.816625 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.816629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:59.817616 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:53:59.817627 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:53:59.818207 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:53:59.818212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:53:59.818515 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:53:59.819297 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:53:59.819327 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:53:59.821436 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:53:59.823597 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:59.823603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:59.823617 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:59.823893 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:59.824273 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:53:59.824276 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:53:59.824277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:53:59.824426 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:53:59.824838 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:53:59.824844 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:53:59.825210 (+ 366us) client_negotiation.cc:769] Sending connection context
0810 18:53:59.825421 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 18:53:59.825702 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"thread_start_us":173,"threads_started":1}
I20260810 18:53:59.828379 11489 heartbeater.cc:347] Connected to a master server at 127.30.153.190:46099
I20260810 18:53:59.828677 11489 heartbeater.cc:464] Registering TS with master...
I20260810 18:53:59.829229 11489 heartbeater.cc:511] Master 127.30.153.190:46099 requested a full tablet report, sending...
I20260810 18:53:59.830389 11007 ts_manager.cc:194] Registered new tserver with Master: 41fefa851c1c47d2b96604fe67d23f1a (127.30.153.130:36365)
I20260810 18:53:59.831105 11007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:39433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:53:59.845495 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:46099
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:53:59.969098 11496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:53:59.969352 11496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:53:59.969394 11496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:53:59.973326 11496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:53:59.973382 11496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:53:59.973438 11496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:53:59.978619 11496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:53:59.979630 11496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:53:59.980607 11496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:53:59.990649 11508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:53:59.990645 11504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:59.990711 11496 server_base.cc:1034] running on GCE node
W20260810 18:53:59.990643 11505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:53:59.991427 11496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:53:59.992087 11496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:53:59.993289 11496 hybrid_clock.cc:648] HybridClock initialized: now 1786388039993267 us; error 33 us; skew 500 ppm
Aug 10 18:53:59 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:53:59.996160 11496 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:53:59.997009 11496 webserver.cc:460] Webserver started at http://127.30.153.131:32835/ using document root <none> and password file <none>
I20260810 18:53:59.997597 11496 fs_manager.cc:359] Metadata directory not provided
I20260810 18:53:59.997692 11496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:53:59.997925 11496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:00.000054 11496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "a90ec5219d28411abc5e1b9c85784a43"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "4b1b064e0f7bdda6db0aa776e00f8254"
server_key_iv: "8c7f4a5fbfa5db641a3c145e73f17f26"
server_key_version: "encryptionkey@0"
I20260810 18:54:00.000680 11496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "a90ec5219d28411abc5e1b9c85784a43"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "4b1b064e0f7bdda6db0aa776e00f8254"
server_key_iv: "8c7f4a5fbfa5db641a3c145e73f17f26"
server_key_version: "encryptionkey@0"
I20260810 18:54:00.005858 11496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:54:00.009408 11517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.011482 11496 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:54:00.011672 11496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "a90ec5219d28411abc5e1b9c85784a43"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "4b1b064e0f7bdda6db0aa776e00f8254"
server_key_iv: "8c7f4a5fbfa5db641a3c145e73f17f26"
server_key_version: "encryptionkey@0"
I20260810 18:54:00.011828 11496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:00.025463 11496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:00.028966 11496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:00.029201 11496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:00.029852 11496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:00.030800 11496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:00.030874 11496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.030946 11496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:00.030989 11496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.040899 11496 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:33201
I20260810 18:54:00.040927 11691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:33201 every 8 connection(s)
I20260810 18:54:00.041838 11496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:54:00.043272 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 11496
I20260810 18:54:00.043437 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:54:00.043795 31334 external_mini_cluster.cc:1428] Setting key 61312c642551f78cf1208d5cca25a87e
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388039, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:54:00.057257 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:00.044214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:37875 (local address 127.30.153.190:46099)
0810 18:54:00.044375 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:00.044382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:00.045733 (+ 1351us) server_negotiation.cc:407] Connection header received
0810 18:54:00.046938 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:00.046941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:00.046991 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:00.047064 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:00.048289 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.048868 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.050018 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.050288 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.050379 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.053188 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:00.053199 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:00.053203 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:00.053230 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:00.054836 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:00.055412 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:00.055417 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:00.055419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:00.055511 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:00.055922 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:00.055926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:00.055928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:00.056137 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:00.056257 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:00.056906 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 18:54:00.057044 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:54:00.058311 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:00.044663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:37875)
0810 18:54:00.045568 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:00.045623 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:00.046719 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:00.047262 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:00.047269 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:00.047657 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:00.048136 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.048148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.048981 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.048983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.049896 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.049906 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.050475 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.050480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.050639 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.051207 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:00.051223 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:00.052971 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:00.054966 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:00.054972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:00.054989 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:00.055293 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:00.055646 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:00.055649 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:00.055652 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:00.055804 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:00.056257 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:00.056265 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:00.056606 (+ 341us) client_negotiation.cc:769] Sending connection context
0810 18:54:00.056861 (+ 255us) client_negotiation.cc:240] Negotiation successful
0810 18:54:00.057221 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":176,"threads_started":1}
I20260810 18:54:00.059516 11693 heartbeater.cc:347] Connected to a master server at 127.30.153.190:46099
I20260810 18:54:00.059814 11693 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:00.060388 11693 heartbeater.cc:511] Master 127.30.153.190:46099 requested a full tablet report, sending...
I20260810 18:54:00.061667 11007 ts_manager.cc:194] Registered new tserver with Master: a90ec5219d28411abc5e1b9c85784a43 (127.30.153.131:33201)
I20260810 18:54:00.062320 11007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:37875
I20260810 18:54:00.070363 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab.
Entry for principal oryx/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab.
WARNING: no policy specified for oryx/127.30.153.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab.
Entry for principal oryx/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab.
WARNING: no policy specified for oryx/127.30.153.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab.
Entry for principal oryx/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab.
I20260810 18:54:00.162315 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 11097
I20260810 18:54:00.181417 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 11283
I20260810 18:54:00.191015 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 11496
I20260810 18:54:00.202785 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 10966
I20260810 18:54:00.216459 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:46099
--webserver_interface=127.30.153.190
--webserver_port=41297
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:46099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:00.335493 11704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:00.335783 11704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:00.335824 11704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:00.339886 11704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:00.339946 11704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:00.339964 11704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:00.339982 11704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:00.339996 11704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:00.345389 11704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:46099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:46099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:00.346530 11704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:00.347612 11704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:00.356869 11714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:00.356869 11711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:00.357013 11704 server_base.cc:1034] running on GCE node
W20260810 18:54:00.356871 11718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:00.357951 11704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:00.359241 11704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:00.360420 11704 hybrid_clock.cc:648] HybridClock initialized: now 1786388040360404 us; error 30 us; skew 500 ppm
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388040, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:00.363369 11704 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:54:00.364298 11704 webserver.cc:460] Webserver started at http://127.30.153.190:41297/ using document root <none> and password file <none>
I20260810 18:54:00.364849 11704 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:00.364924 11704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:00.368937 11704 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:54:00.371155 11727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.372452 11704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:54:00.372583 11704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "7d9f16edc4be404e989d3c37262bceb0"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "ccac1c6ecf264a5ce22d1e8d613cc919"
server_key_iv: "4cdbfb1e61e554d01676273dd4b029d7"
server_key_version: "encryptionkey@0"
I20260810 18:54:00.373212 11704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:00.398658 11704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:00.401832 11704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:00.402058 11704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:00.409642 11704 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:46099
I20260810 18:54:00.409684 11805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:46099 every 8 connection(s)
I20260810 18:54:00.410753 11704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:00.413906 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 11704
I20260810 18:54:00.414474 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:33569
--local_ip_for_outbound_sockets=127.30.153.129
--tserver_master_addrs=127.30.153.190:46099
--webserver_port=35277
--webserver_interface=127.30.153.129
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.153.129
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:54:00.415666 11806 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:54:00.417650 11806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Bootstrap starting.
I20260810 18:54:00.435707 11806 log.cc:824] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:00.442286 11806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:54:00.442692 11806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Bootstrap complete.
I20260810 18:54:00.447239 11806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } }
I20260810 18:54:00.447635 11806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d9f16edc4be404e989d3c37262bceb0, State: Initialized, Role: FOLLOWER
I20260810 18:54:00.448120 11806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } }
I20260810 18:54:00.448236 11806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:00.448328 11806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:00.448446 11806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:54:00.450513 11806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } }
I20260810 18:54:00.450841 11806 leader_election.cc:302] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d9f16edc4be404e989d3c37262bceb0; no voters:
I20260810 18:54:00.451188 11806 leader_election.cc:288] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:54:00.451318 11811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:54:00.451570 11811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [term 2 LEADER]: Becoming Leader. State: Replica: 7d9f16edc4be404e989d3c37262bceb0, State: Running, Role: LEADER
I20260810 18:54:00.451951 11811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } }
I20260810 18:54:00.452420 11806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:00.453504 11815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d9f16edc4be404e989d3c37262bceb0. Latest consensus state: current_term: 2 leader_uuid: "7d9f16edc4be404e989d3c37262bceb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } } }
I20260810 18:54:00.453662 11815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:00.453532 11814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7d9f16edc4be404e989d3c37262bceb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d9f16edc4be404e989d3c37262bceb0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 46099 } } }
I20260810 18:54:00.453974 11814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:00.454047 11833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:00.457299 11833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:00.457957 11833 catalog_manager.cc:1259] Loaded cluster ID: df003bf528c14bf699f90249ac0dae3e
I20260810 18:54:00.458050 11833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:00.460477 11833 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:00.461282 11833 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d9f16edc4be404e989d3c37262bceb0: Loaded TSK: 0
I20260810 18:54:00.462797 11833 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:54:00.533815 11808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:00.534094 11808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:00.534171 11808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:00.538199 11808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:00.538290 11808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:00.538385 11808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:54:00.543679 11808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.129.keytab
--principal=oryx/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:33569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=35277
--tserver_master_addrs=127.30.153.190:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:00.544692 11808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:00.545693 11808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:00.556165 11857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:00.556213 11853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:00.556437 11855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:00.556560 11808 server_base.cc:1034] running on GCE node
I20260810 18:54:00.557216 11808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:00.557875 11808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:00.559055 11808 hybrid_clock.cc:648] HybridClock initialized: now 1786388040559070 us; error 72 us; skew 500 ppm
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388040, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:00.561849 11808 init.cc:377] Logged in from keytab as oryx/127.30.153.129@KRBTEST.COM (short username oryx)
I20260810 18:54:00.562868 11808 webserver.cc:460] Webserver started at http://127.30.153.129:35277/ using document root <none> and password file <none>
I20260810 18:54:00.563431 11808 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:00.563506 11808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:00.567463 11808 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:54:00.569622 11865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.570977 11808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:54:00.571106 11808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "12093e00756e44b5bfe854578147396d"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "0c20bcbbfcf927f42e1fc7ef1cb252a8"
server_key_iv: "98c16bdb8c3eaa09802fb147e7b7dad2"
server_key_version: "encryptionkey@0"
I20260810 18:54:00.571715 11808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:00.585153 11808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:00.588581 11808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:00.588792 11808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:00.589463 11808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:00.590405 11808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:00.590512 11808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.590581 11808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:00.590613 11808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.601796 11808 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:33569
I20260810 18:54:00.601823 12036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:33569 every 8 connection(s)
I20260810 18:54:00.603029 11808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:54:00.612663 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 11808
I20260810 18:54:00.613197 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:36365
--local_ip_for_outbound_sockets=127.30.153.130
--tserver_master_addrs=127.30.153.190:46099
--webserver_port=35699
--webserver_interface=127.30.153.130
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.153.130
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:54:00.617522 12042 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:00.605417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:48367 (local address 127.30.153.190:46099)
0810 18:54:00.605950 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:00.605972 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:00.606651 (+ 679us) server_negotiation.cc:407] Connection header received
0810 18:54:00.607738 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:00.607750 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:00.608088 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:00.608497 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:00.609632 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.610750 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.611943 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.612632 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.612842 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.616752 (+ 3910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:48367: BlockingRecv error: recv got EOF from 127.30.153.129:48367 (error 108)
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":121,"threads_started":1}
I20260810 18:54:00.617537 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:00.605585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:48367)
0810 18:54:00.606404 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:00.606519 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:00.607324 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:00.608595 (+ 1271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:00.608601 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:00.608958 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:00.609429 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.609443 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.610891 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.610894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.611806 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.611816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.612861 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.612868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.613028 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.613799 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:00.613816 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:00.616519 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":161,"threads_started":1}
W20260810 18:54:00.618052 12037 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:46099 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.153.190:46099: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:00.625617 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:00.618647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:55385)
0810 18:54:00.618852 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:00.618868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:00.618957 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:00.619241 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:00.619247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:00.619449 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:00.619628 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.619639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.620628 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.620635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.621666 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.621681 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.622232 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.622238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.622370 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.622950 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:00.622972 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:00.625416 (+ 2444us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117}
W20260810 18:54:00.625732 12042 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:00.618740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:55385 (local address 127.30.153.190:46099)
0810 18:54:00.619015 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:00.619018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:00.619030 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:54:00.619066 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:00.619068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:00.619107 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:00.619184 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:00.619831 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.620472 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.621827 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.622065 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.622199 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.625595 (+ 3396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:55385: BlockingRecv error: recv got EOF from 127.30.153.129:55385 (error 108)
Metrics: {"server-negotiator.queue_time_us":209}
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:00.632041 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:00.626444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:48551)
0810 18:54:00.626578 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:00.626598 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:00.626690 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:00.626977 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:00.626980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:00.627195 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:00.627293 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.627299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.628102 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.628107 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.628688 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.628696 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.629266 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.629270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.629368 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.629900 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:00.629913 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:00.631878 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 18:54:00.632133 12042 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:00.626520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:48551 (local address 127.30.153.190:46099)
0810 18:54:00.626678 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:00.626682 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:00.626698 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:54:00.626797 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:00.626802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:00.626849 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:00.626945 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:00.627426 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.627974 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.628833 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.629140 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.629262 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.632005 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:48551: BlockingRecv error: recv got EOF from 127.30.153.129:48551 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260810 18:54:00.632557 12037 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:54:00.732880 12044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:00.733196 12044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:00.733280 12044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:00.737300 12044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:00.737393 12044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:00.737488 12044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:54:00.742700 12044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.130.keytab
--principal=oryx/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:36365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=35699
--tserver_master_addrs=127.30.153.190:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:00.743790 12044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:00.744841 12044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:00.755445 12052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:00.755475 12050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:00.755445 12049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:00.755788 12044 server_base.cc:1034] running on GCE node
I20260810 18:54:00.756312 12044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:00.756963 12044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:00.758162 12044 hybrid_clock.cc:648] HybridClock initialized: now 1786388040758148 us; error 28 us; skew 500 ppm
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388040, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:00.760998 12044 init.cc:377] Logged in from keytab as oryx/127.30.153.130@KRBTEST.COM (short username oryx)
I20260810 18:54:00.761991 12044 webserver.cc:460] Webserver started at http://127.30.153.130:35699/ using document root <none> and password file <none>
I20260810 18:54:00.762640 12044 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:00.762750 12044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:00.766750 12044 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:54:00.768922 12063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.770403 12044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:54:00.770629 12044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "41fefa851c1c47d2b96604fe67d23f1a"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "d21c1d53cfbd091310d0251755e97648"
server_key_iv: "d5bffbfda68a0e0d335ef1ee32113432"
server_key_version: "encryptionkey@0"
I20260810 18:54:00.771270 12044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:00.789378 12044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:00.792658 12044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:00.792888 12044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:00.793529 12044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:00.794468 12044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:00.794529 12044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.794576 12044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:00.794631 12044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.804208 12044 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:36365
I20260810 18:54:00.804239 12241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:36365 every 8 connection(s)
I20260810 18:54:00.805301 12044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:54:00.812070 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 12044
I20260810 18:54:00.812536 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:33201
--local_ip_for_outbound_sockets=127.30.153.131
--tserver_master_addrs=127.30.153.190:46099
--webserver_port=32835
--webserver_interface=127.30.153.131
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.153.131
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 18:54:00.817093 12042 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:00.807275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:42583 (local address 127.30.153.190:46099)
0810 18:54:00.807421 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:00.807424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:00.808533 (+ 1109us) server_negotiation.cc:407] Connection header received
0810 18:54:00.809568 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:00.809572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:00.809640 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:00.809759 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:00.810977 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.811532 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.813075 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.813310 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.813449 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.816963 (+ 3514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:42583: BlockingRecv error: recv got EOF from 127.30.153.130:42583 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:54:00.817750 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:00.807616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:42583)
0810 18:54:00.808362 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:00.808404 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:00.809342 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:00.809940 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:00.809948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:00.810305 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:00.810814 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.810828 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.811670 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.811674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.812928 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.812939 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.813488 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.813492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.813672 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.814311 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:00.814327 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:00.816820 (+ 2493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":119,"threads_started":1}
W20260810 18:54:00.818238 12243 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:46099 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.153.190:46099: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:00.824404 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:00.818810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:59011)
0810 18:54:00.818948 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:00.818964 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:00.819052 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:00.819343 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:00.819346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:00.819513 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:00.819602 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.819607 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.820360 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.820365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.820917 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.820926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.821535 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.821539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.821659 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.822188 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:00.822200 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:00.824213 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 18:54:00.824476 12042 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:00.818893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:59011 (local address 127.30.153.190:46099)
0810 18:54:00.819018 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:00.819022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:00.819036 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:54:00.819170 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:00.819173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:00.819229 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:00.819325 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:00.819731 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.820247 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.821165 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.821395 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.821519 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.824351 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:59011: BlockingRecv error: recv got EOF from 127.30.153.130:59011 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:00.830361 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:00.824980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:33445)
0810 18:54:00.825137 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:00.825154 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:00.825240 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:00.825514 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:00.825517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:00.825685 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:00.825756 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.825762 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.826560 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.826563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.827080 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:00.827088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.827549 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.827553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:00.827641 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.828158 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:00.828169 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:00.830214 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 18:54:00.830569 12042 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:00.825177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:33445 (local address 127.30.153.190:46099)
0810 18:54:00.825304 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:00.825311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:00.825325 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:54:00.825368 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:00.825370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:00.825403 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:00.825495 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:00.825880 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.826389 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.827212 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:00.827429 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:00.827540 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:00.830442 (+ 2902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:33445: BlockingRecv error: recv got EOF from 127.30.153.130:33445 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 18:54:00.830821 12243 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:54:00.932565 12247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:00.932827 12247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:00.932870 12247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:00.936822 12247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:00.936878 12247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:00.936933 12247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:54:00.942078 12247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/krb5kdc/oryx_127.30.153.131.keytab
--principal=oryx/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:33201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=32835
--tserver_master_addrs=127.30.153.190:46099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:00.943056 12247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:00.944015 12247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:00.954145 12255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:00.954159 12258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:00.954238 12264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:00.954808 12247 server_base.cc:1034] running on GCE node
I20260810 18:54:00.955258 12247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:00.955906 12247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:00.957072 12247 hybrid_clock.cc:648] HybridClock initialized: now 1786388040957057 us; error 32 us; skew 500 ppm
Aug 10 18:54:00 dist-test-slave-68xk krb5kdc[10946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388040, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:00.960032 12247 init.cc:377] Logged in from keytab as oryx/127.30.153.131@KRBTEST.COM (short username oryx)
I20260810 18:54:00.960914 12247 webserver.cc:460] Webserver started at http://127.30.153.131:32835/ using document root <none> and password file <none>
I20260810 18:54:00.961488 12247 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:00.961604 12247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:00.965576 12247 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:54:00.967801 12275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:00.969081 12247 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:54:00.969234 12247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "a90ec5219d28411abc5e1b9c85784a43"
format_stamp: "Formatted at 2026-08-10 18:53:59 on dist-test-slave-68xk"
server_key: "4b1b064e0f7bdda6db0aa776e00f8254"
server_key_iv: "8c7f4a5fbfa5db641a3c145e73f17f26"
server_key_version: "encryptionkey@0"
I20260810 18:54:00.969839 12247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:01.003007 12247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:01.007347 12247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:01.007618 12247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:01.008528 12247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:01.009693 12247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:01.009758 12247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:01.009819 12247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:01.009851 12247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:01.019763 12247 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:33201
I20260810 18:54:01.019790 12444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:33201 every 8 connection(s)
I20260810 18:54:01.020802 12247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:01.030839 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 12247
W20260810 18:54:01.031893 12042 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:01.022700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:58179 (local address 127.30.153.190:46099)
0810 18:54:01.022857 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:01.022863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:01.023990 (+ 1127us) server_negotiation.cc:407] Connection header received
0810 18:54:01.024829 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:01.024832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:01.024881 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:01.024962 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:01.026189 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.026811 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.027906 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.028149 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.028233 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.031734 (+ 3501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:58179: BlockingRecv error: recv got EOF from 127.30.153.131:58179 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:54:01.032639 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:01.023177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:58179)
0810 18:54:01.023837 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:01.023878 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:01.024633 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:01.025150 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:01.025157 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:01.025516 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:01.026001 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.026014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.026928 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.026931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.027789 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.027798 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.028281 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.028284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.028419 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.028969 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:01.028985 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:01.031598 (+ 2613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":111,"threads_started":1}
W20260810 18:54:01.033102 12446 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:46099 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.153.190:46099: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:01.039192 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:01.033563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:46689)
0810 18:54:01.033683 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:01.033700 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:01.033774 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:01.034142 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:01.034145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:01.034266 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:01.034355 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.034360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.035119 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.035122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.035839 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.035849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.036442 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.036446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.036548 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.036952 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:01.036970 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:01.039047 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 18:54:01.039328 12452 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:01.033643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:46689 (local address 127.30.153.190:46099)
0810 18:54:01.033929 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:01.033933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:01.033945 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:54:01.033978 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:01.033981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:01.034020 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:01.034106 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:01.034502 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.035001 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.035965 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.036310 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.036412 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.039188 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:46689: BlockingRecv error: recv got EOF from 127.30.153.131:46689 (error 108)
Metrics: {"server-negotiator.queue_time_us":214}
I20260810 18:54:01.042235 12042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:01.031862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.30.153.190:46099)
0810 18:54:01.032067 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:01.032072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:01.032108 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:54:01.032143 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:01.032146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:01.032184 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:01.032264 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:01.033727 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.034393 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.035433 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.035636 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.035713 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.037510 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:01.037515 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:01.037529 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:01.037581 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:01.039863 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:01.040487 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:01.040492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:01.040497 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:01.040672 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:01.040953 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:01.040956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:01.040958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:01.041403 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:01.041578 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:01.041844 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 18:54:01.042081 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:01.045511 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:01.039729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:46197)
0810 18:54:01.039838 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:01.039853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:01.039948 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:01.040179 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:01.040182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:01.040361 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:01.040472 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.040479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.041442 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.041445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.042198 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.042207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.042759 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.042762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.042818 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.043289 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:01.043300 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:01.045427 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:54:01.045723 12446 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:54:01.045886 12452 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:01.039804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:46197 (local address 127.30.153.190:46099)
0810 18:54:01.039960 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:01.039965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:01.039978 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:54:01.040020 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:01.040023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:01.040066 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:01.040130 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:01.040579 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.041323 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.042315 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.042644 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.042747 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.045535 (+ 2788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:46197: BlockingRecv error: recv got EOF from 127.30.153.131:46197 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:01.639336 12453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:01.633332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:54857)
0810 18:54:01.633724 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:01.633742 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:01.633863 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:01.634082 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:01.634084 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:01.634285 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:01.634437 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.634444 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.635245 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.635248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.635833 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.635841 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.636368 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.636371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.636465 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.637064 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:01.637077 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:01.639220 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":88,"threads_started":1}
W20260810 18:54:01.639567 12454 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:01.633386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:54857 (local address 127.30.153.190:46099)
0810 18:54:01.633722 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:01.633727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:01.633803 (+ 76us) server_negotiation.cc:407] Connection header received
0810 18:54:01.633939 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:01.633941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:01.633989 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:01.634074 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:01.634544 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.635114 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.635949 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.636224 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.636319 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.639411 (+ 3092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:54857: BlockingRecv error: recv got EOF from 127.30.153.129:54857 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":110,"threads_started":1}
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:01 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:01.838025 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:01.831577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:37539)
0810 18:54:01.831907 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:01.831926 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:01.832076 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:01.832398 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:01.832401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:01.832617 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:01.832742 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.832749 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.833615 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.833620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.834377 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:01.834388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.834956 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.834959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:01.835034 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.835614 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:01.835645 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:01.837906 (+ 2261us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":102,"threads_started":1}
W20260810 18:54:01.838383 12454 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:01.831651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:37539 (local address 127.30.153.190:46099)
0810 18:54:01.831873 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:01.831878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:01.832015 (+ 137us) server_negotiation.cc:407] Connection header received
0810 18:54:01.832228 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:01.832231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:01.832276 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:01.832369 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:01.832868 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.833503 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.834547 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:01.834843 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:01.834936 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:01.838097 (+ 3161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:37539: BlockingRecv error: recv got EOF from 127.30.153.130:37539 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 10 18:54:02 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:02 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:02.052081 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:02.046419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:58983)
0810 18:54:02.046715 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:02.046729 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:02.046825 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:02.047066 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:02.047069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:02.047246 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:02.047373 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:02.047378 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.048156 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.048158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:02.048739 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:02.048747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.049215 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.049218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:02.049293 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:02.049863 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:02.049875 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:02.051890 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":96,"threads_started":1}
W20260810 18:54:02.052160 12454 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:02.046538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:58983 (local address 127.30.153.190:46099)
0810 18:54:02.046695 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:02.046699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:02.046783 (+ 84us) server_negotiation.cc:407] Connection header received
0810 18:54:02.046891 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:02.046893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:02.046939 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:02.047013 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:02.047484 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.048051 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.048866 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.049108 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.049230 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:02.052033 (+ 2803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:58983: BlockingRecv error: recv got EOF from 127.30.153.131:58983 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 18:54:02 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:02 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:02.646713 12457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:02.640191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:56069)
0810 18:54:02.640562 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:02.640579 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:02.640685 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:02.641145 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:02.641150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:02.641398 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:02.641514 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:02.641521 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.642323 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.642326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:02.642999 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:02.643009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.643717 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.643720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:02.643786 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:02.644298 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:02.644312 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:02.646516 (+ 2204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":110,"threads_started":1}
W20260810 18:54:02.646833 12458 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:02.640382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:56069 (local address 127.30.153.190:46099)
0810 18:54:02.640634 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:02.640638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:02.640652 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:54:02.640829 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:02.640832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:02.640893 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:02.640999 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:02.641648 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.642183 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.643113 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.643580 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.643677 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:02.646656 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:56069: BlockingRecv error: recv got EOF from 127.30.153.129:56069 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":102,"threads_started":1}
Aug 10 18:54:02 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:02 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:02.848964 12459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:02.838977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:34841)
0810 18:54:02.839273 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:02.839291 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:02.839392 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:02.839681 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:02.839683 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:02.839887 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:02.839991 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:02.839997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.840808 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.840810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:02.841378 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:02.841386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.841884 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.841886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:02.841947 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:02.846798 (+ 4851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:02.846811 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:02.848826 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":113,"threads_started":1}
W20260810 18:54:02.849177 12458 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:02.839066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:34841 (local address 127.30.153.190:46099)
0810 18:54:02.839238 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:02.839242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:02.839345 (+ 103us) server_negotiation.cc:407] Connection header received
0810 18:54:02.839459 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:02.839461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:02.839504 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:02.839583 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:02.840141 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.840697 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.841520 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:02.841771 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:02.841870 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:02.849031 (+ 7161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:34841: BlockingRecv error: recv got EOF from 127.30.153.130:34841 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 18:54:03 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:03 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:03.059039 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:03.052942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:41873)
0810 18:54:03.053326 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:03.053345 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:03.053505 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:03.053805 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:03.053808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:03.053996 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:03.054100 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:03.054105 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.055114 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.055117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:03.055666 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:03.055674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.056156 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.056159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:03.056245 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:03.056757 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:03.056769 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:03.058913 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":95,"threads_started":1}
W20260810 18:54:03.059247 12458 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:03.053092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:41873 (local address 127.30.153.190:46099)
0810 18:54:03.053291 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:03.053296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:03.053466 (+ 170us) server_negotiation.cc:407] Connection header received
0810 18:54:03.053612 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:03.053618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:03.053679 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:03.053765 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:03.054228 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.054954 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.055794 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.056035 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.056151 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:03.059077 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:41873: BlockingRecv error: recv got EOF from 127.30.153.131:41873 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 18:54:03 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:03 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:03.653977 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:03.647745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:41675)
0810 18:54:03.648066 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:03.648083 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:03.648195 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:03.648517 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:03.648520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:03.648731 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:03.648834 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:03.648840 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.649591 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.649593 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:03.650194 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:03.650202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.650741 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.650743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:03.650835 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:03.651413 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:03.651432 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:03.653813 (+ 2381us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":113,"threads_started":1}
W20260810 18:54:03.654178 12464 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:03.647931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:41675 (local address 127.30.153.190:46099)
0810 18:54:03.648175 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:03.648179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:03.648193 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:54:03.648297 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:03.648302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:03.648363 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:03.648474 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:03.648945 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.649488 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.650310 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.650627 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.650723 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:03.653978 (+ 3255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:41675: BlockingRecv error: recv got EOF from 127.30.153.129:41675 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
Aug 10 18:54:03 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:03 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:03.856951 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:03.849916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:51049)
0810 18:54:03.850299 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:03.850313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:03.850489 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:03.850757 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:03.850759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:03.850965 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:03.851065 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:03.851071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.852067 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.852071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:03.853028 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:03.853040 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.853729 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.853733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:03.853829 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:03.854468 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:03.854490 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:03.856743 (+ 2253us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":181,"threads_started":1}
W20260810 18:54:03.856995 12464 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:03.850005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:51049 (local address 127.30.153.190:46099)
0810 18:54:03.850182 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:03.850188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:03.850503 (+ 315us) server_negotiation.cc:407] Connection header received
0810 18:54:03.850583 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:03.850585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:03.850637 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:03.850716 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:03.851197 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.851947 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.853181 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:03.853570 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:03.853715 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:03.856884 (+ 3169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:51049: BlockingRecv error: recv got EOF from 127.30.153.130:51049 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 18:54:04 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:04 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:04.065953 12466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:04.059830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:38499)
0810 18:54:04.060109 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:04.060122 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:04.060230 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:04.060493 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:04.060495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:04.060687 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:04.060785 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:04.060790 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.061654 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.061657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:04.062240 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:04.062250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.062786 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.062788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:04.062857 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:04.063452 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:04.063463 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:04.065823 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":96,"threads_started":1}
W20260810 18:54:04.066107 12464 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:04.059956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:38499 (local address 127.30.153.190:46099)
0810 18:54:04.060104 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:04.060108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:04.060184 (+ 76us) server_negotiation.cc:407] Connection header received
0810 18:54:04.060320 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:04.060323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:04.060374 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:04.060455 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:04.060898 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.061525 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.062356 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.062669 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.062762 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:04.065972 (+ 3210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:38499: BlockingRecv error: recv got EOF from 127.30.153.131:38499 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 18:54:04 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:04 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:04.660665 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:04.654884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:43445)
0810 18:54:04.655227 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:04.655243 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:04.655353 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:04.655626 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:04.655629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:04.655830 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:04.655940 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:04.655945 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.656705 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.656709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:04.657333 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:04.657342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.657800 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.657803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:04.657877 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:04.658385 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:04.658398 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:04.660543 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":96,"threads_started":1}
W20260810 18:54:04.660853 12470 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:04.654931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:43445 (local address 127.30.153.190:46099)
0810 18:54:04.655258 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:04.655262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:04.655319 (+ 57us) server_negotiation.cc:407] Connection header received
0810 18:54:04.655440 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:04.655442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:04.655488 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:04.655568 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:04.656046 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.656581 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.657456 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.657673 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.657760 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:04.660707 (+ 2947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:43445: BlockingRecv error: recv got EOF from 127.30.153.129:43445 (error 108)
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":136,"threads_started":1}
Aug 10 18:54:04 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:04 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:04.863371 12471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:04.857808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:49605)
0810 18:54:04.858182 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:04.858198 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:04.858312 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:04.858606 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:04.858608 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:04.858813 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:04.858919 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:04.858925 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.859701 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.859704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:04.860307 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:04.860320 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.860738 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.860741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:04.860805 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:04.861331 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:04.861343 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:04.863245 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":126,"threads_started":1}
W20260810 18:54:04.863554 12470 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:04.857851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:49605 (local address 127.30.153.190:46099)
0810 18:54:04.858010 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:04.858015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:04.858278 (+ 263us) server_negotiation.cc:407] Connection header received
0810 18:54:04.858394 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:04.858396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:04.858491 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:04.858562 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:04.859024 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.859594 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.860423 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:04.860635 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:04.860720 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:04.863397 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:49605: BlockingRecv error: recv got EOF from 127.30.153.130:49605 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 18:54:05 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:05 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:05.072185 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:05.066833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:56683)
0810 18:54:05.067137 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:05.067156 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:05.067255 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:05.067506 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:05.067508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:05.067697 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:05.067796 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:05.067801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.068556 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.068558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:05.069103 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:05.069111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.069521 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.069524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:05.069585 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:05.070069 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:05.070082 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:05.072063 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":135,"threads_started":1}
W20260810 18:54:05.072451 12470 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:05.066891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:56683 (local address 127.30.153.190:46099)
0810 18:54:05.067065 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:05.067069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:05.067224 (+ 155us) server_negotiation.cc:407] Connection header received
0810 18:54:05.067341 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:05.067344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:05.067399 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:05.067468 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:05.067897 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.068456 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.069204 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.069424 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.069503 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:05.072321 (+ 2818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:56683: BlockingRecv error: recv got EOF from 127.30.153.131:56683 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 18:54:05 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:05 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:05.667496 12473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:05.661638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:60955)
0810 18:54:05.661954 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:05.661976 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:05.662082 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:05.662365 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:05.662368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:05.662595 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:05.662704 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:05.662710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.663487 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.663490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:05.664122 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:05.664138 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.664564 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.664566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:05.664634 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:05.665250 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:05.665264 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:05.667368 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":96,"threads_started":1}
W20260810 18:54:05.667630 12475 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:05.661804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:60955 (local address 127.30.153.190:46099)
0810 18:54:05.662058 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:05.662062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:05.662077 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:54:05.662172 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:05.662175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:05.662219 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:05.662319 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:05.662810 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.663375 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.664230 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.664463 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.664543 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:05.667495 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:60955: BlockingRecv error: recv got EOF from 127.30.153.129:60955 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":87,"threads_started":1}
Aug 10 18:54:05 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:05 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:05.870977 12477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:05.864331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:55865)
0810 18:54:05.864681 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:05.864698 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:05.864817 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:05.865112 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:05.865115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:05.865350 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:05.865443 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:05.865449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.866225 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.866230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:05.867018 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:05.867029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.867479 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.867483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:05.867569 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:05.868146 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:05.868160 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:05.870853 (+ 2693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":123,"threads_started":1}
W20260810 18:54:05.871102 12475 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:05.864538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:55865 (local address 127.30.153.190:46099)
0810 18:54:05.864701 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:05.864707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:05.864761 (+ 54us) server_negotiation.cc:407] Connection header received
0810 18:54:05.864942 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:05.864945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:05.864990 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:05.865112 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:05.865561 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.866112 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.867145 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:05.867365 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:05.867475 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:05.870985 (+ 3510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:55865: BlockingRecv error: recv got EOF from 127.30.153.130:55865 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 10 18:54:06 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:06 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:06.079682 12478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:06.073052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:35157)
0810 18:54:06.073341 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:06.073355 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:06.073469 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:06.073785 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:06.073792 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:06.074016 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:06.074124 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:06.074132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.075142 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.075144 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:06.075909 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:06.075918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.076447 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.076450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:06.076522 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:06.077129 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:06.077141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:06.079503 (+ 2362us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":79,"threads_started":1}
W20260810 18:54:06.079824 12475 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:06.073208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:35157 (local address 127.30.153.190:46099)
0810 18:54:06.073359 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:06.073363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:06.073461 (+ 98us) server_negotiation.cc:407] Connection header received
0810 18:54:06.073544 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:06.073547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:06.073597 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:06.073677 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:06.074257 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.074989 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.076038 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.076310 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.076409 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:06.079676 (+ 3267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:35157: BlockingRecv error: recv got EOF from 127.30.153.131:35157 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 18:54:06 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:06 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:06.674321 12479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:06.668468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:58765)
0810 18:54:06.668774 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:06.668788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:06.668883 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:06.669188 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:06.669192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:06.669421 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:06.669539 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:06.669549 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.670591 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.670594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:06.671167 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:06.671176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.671620 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.671623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:06.671693 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:06.672211 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:06.672224 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:06.674186 (+ 1962us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":80,"threads_started":1}
W20260810 18:54:06.674547 12482 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:06.668535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:58765 (local address 127.30.153.190:46099)
0810 18:54:06.668825 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:06.668829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:06.668864 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:54:06.668974 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:06.668977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:06.669037 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:06.669166 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:06.669667 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.670484 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.671270 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.671487 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.671565 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:06.674356 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:58765: BlockingRecv error: recv got EOF from 127.30.153.129:58765 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":104,"threads_started":1}
Aug 10 18:54:06 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:06 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:06.878456 12483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:06.871814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:60467)
0810 18:54:06.872107 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:06.872121 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:06.872225 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:06.872479 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:06.872481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:06.872703 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:06.872831 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:06.872859 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.873749 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.873755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:06.874516 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:06.874528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.875249 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.875253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:06.875362 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:06.875983 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:06.875996 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:06.878238 (+ 2242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":107,"threads_started":1}
W20260810 18:54:06.878535 12482 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:06.871918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:60467 (local address 127.30.153.190:46099)
0810 18:54:06.872066 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:06.872070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:06.872187 (+ 117us) server_negotiation.cc:407] Connection header received
0810 18:54:06.872304 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:06.872306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:06.872353 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:06.872425 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:06.872993 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.873621 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.874659 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:06.875079 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:06.875213 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:06.878374 (+ 3161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:60467: BlockingRecv error: recv got EOF from 127.30.153.130:60467 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 18:54:07 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:07 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:07.086257 12484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:07.080588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:38545)
0810 18:54:07.080875 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:07.080889 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:07.080996 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:07.081339 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:07.081341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:07.081539 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:07.081635 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:07.081640 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.082474 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.082478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:07.083031 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:07.083039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.083504 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.083507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:07.083567 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:07.084114 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:07.084127 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:07.086085 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":96,"threads_started":1}
W20260810 18:54:07.086385 12482 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:07.080651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:38545 (local address 127.30.153.190:46099)
0810 18:54:07.080798 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:07.080803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:07.080973 (+ 170us) server_negotiation.cc:407] Connection header received
0810 18:54:07.081126 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:07.081129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:07.081223 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:07.081301 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:07.081752 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.082331 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.083149 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.083403 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.083489 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:07.086270 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:38545: BlockingRecv error: recv got EOF from 127.30.153.131:38545 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 18:54:07 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:07 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:07.681629 12487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:07.675210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:37791)
0810 18:54:07.675606 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:07.675621 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:07.675736 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:07.676178 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:07.676181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:07.676416 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:07.676522 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:07.676528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.677586 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.677589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:07.678239 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:07.678249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.678783 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.678786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:07.678872 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:07.679412 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:07.679425 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:07.681494 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":94,"threads_started":1}
W20260810 18:54:07.681859 12489 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:07.675315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:37791 (local address 127.30.153.190:46099)
0810 18:54:07.675629 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:07.675632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:07.675692 (+ 60us) server_negotiation.cc:407] Connection header received
0810 18:54:07.675890 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:07.675894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:07.675952 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:07.676135 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:07.676710 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.677470 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.678386 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.678660 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.678776 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:07.681700 (+ 2924us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:37791: BlockingRecv error: recv got EOF from 127.30.153.129:37791 (error 108)
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":104,"threads_started":1}
Aug 10 18:54:07 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:07 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:07.893419 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:07.879362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:42129)
0810 18:54:07.879788 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:07.879802 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:07.879944 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:07.880226 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:07.880229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:07.880454 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:07.880551 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:07.880556 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.881354 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.881360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:07.881929 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:07.881937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.882465 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.882468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:07.882563 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:07.891232 (+ 8669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:07.891246 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:07.893293 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":119,"threads_started":1}
W20260810 18:54:07.893630 12489 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:07.879517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:42129 (local address 127.30.153.190:46099)
0810 18:54:07.879729 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:07.879735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:07.879884 (+ 149us) server_negotiation.cc:407] Connection header received
0810 18:54:07.880046 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:07.880049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:07.880109 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:07.880198 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:07.880690 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.881238 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.882055 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:07.882304 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:07.882417 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:07.893494 (+ 11077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:42129: BlockingRecv error: recv got EOF from 127.30.153.130:42129 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 10 18:54:08 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:08 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:08.092906 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:08.087118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:49993)
0810 18:54:08.087441 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:08.087455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:08.087552 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:08.087802 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:08.087804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:08.087993 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:08.088090 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:08.088095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.088994 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.088997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:08.089574 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:08.089583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.090040 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.090042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:08.090106 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:08.090695 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:08.090707 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:08.092746 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":102,"threads_started":1}
W20260810 18:54:08.093061 12489 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:08.087282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:49993 (local address 127.30.153.190:46099)
0810 18:54:08.087436 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:08.087440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:08.087524 (+ 84us) server_negotiation.cc:407] Connection header received
0810 18:54:08.087633 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:08.087636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:08.087686 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:08.087767 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:08.088256 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.088871 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.089691 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.089937 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.090025 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:08.092916 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:49993: BlockingRecv error: recv got EOF from 127.30.153.131:49993 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 18:54:08 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:08 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:08.688380 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:08.682609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:42221)
0810 18:54:08.683080 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:08.683096 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:08.683233 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:08.683572 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:08.683574 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:08.683771 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:08.683873 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:08.683879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.684682 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.684685 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:08.685280 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:08.685289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.685749 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.685752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:08.685832 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:08.686368 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:08.686380 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:08.688261 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":113,"threads_started":1}
W20260810 18:54:08.688663 12494 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:08.682685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:42221 (local address 127.30.153.190:46099)
0810 18:54:08.682970 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:08.682974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:08.683182 (+ 208us) server_negotiation.cc:407] Connection header received
0810 18:54:08.683381 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:08.683385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:08.683451 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:08.683530 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:08.683979 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.684548 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.685406 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.685640 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.685723 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:08.688414 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:42221: BlockingRecv error: recv got EOF from 127.30.153.129:42221 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":89,"threads_started":1}
Aug 10 18:54:08 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:08 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:08.899926 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:08.894303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:38559)
0810 18:54:08.894583 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:08.894598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:08.894711 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:08.894935 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:08.894937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:08.895121 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:08.895259 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:08.895266 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.896126 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.896129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:08.896714 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:08.896723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.897135 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.897138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:08.897237 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:08.897775 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:08.897788 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:08.899805 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":116,"threads_started":1}
W20260810 18:54:08.900099 12494 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:08.894404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:38559 (local address 127.30.153.190:46099)
0810 18:54:08.894571 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:08.894575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:08.894657 (+ 82us) server_negotiation.cc:407] Connection header received
0810 18:54:08.894784 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:08.894786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:08.894828 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:08.894899 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:08.895360 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.896021 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.896814 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:08.897034 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:08.897107 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:08.899928 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:38559: BlockingRecv error: recv got EOF from 127.30.153.130:38559 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 10 18:54:09 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:09 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:09.099875 12496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:09.093938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:59453)
0810 18:54:09.094238 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:09.094252 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:09.094347 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:09.094660 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:09.094662 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:09.094857 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:09.094953 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:09.094958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.095806 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.095809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:09.096379 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:09.096387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.096863 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.096866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:09.096928 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:09.097449 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:09.097461 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:09.099683 (+ 2222us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
W20260810 18:54:09.099949 12494 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:09.094113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:59453 (local address 127.30.153.190:46099)
0810 18:54:09.094292 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:09.094296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:09.094312 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:54:09.094482 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:09.094486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:09.094548 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:09.094626 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:09.095107 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.095692 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.096502 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.096745 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.096832 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:09.099822 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:59453: BlockingRecv error: recv got EOF from 127.30.153.131:59453 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 10 18:54:09 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:09 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:09.694965 12497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:09.689359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:43027)
0810 18:54:09.689686 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:09.689700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:09.689801 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:09.690021 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:09.690023 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:09.690214 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:09.690312 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:09.690317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.691157 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.691161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:09.691744 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:09.691752 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.692178 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.692180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:09.692244 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:09.692763 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:09.692776 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:09.694848 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":139,"threads_started":1}
W20260810 18:54:09.695149 12499 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:09.689417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:43027 (local address 127.30.153.190:46099)
0810 18:54:09.689661 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:09.689664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:09.689778 (+ 114us) server_negotiation.cc:407] Connection header received
0810 18:54:09.689883 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:09.689885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:09.689932 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:09.690012 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:09.690475 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.691040 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.691854 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.692078 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.692156 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:09.695000 (+ 2844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:43027: BlockingRecv error: recv got EOF from 127.30.153.129:43027 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
Aug 10 18:54:09 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:09 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:09.906586 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:09.900831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:38505)
0810 18:54:09.901137 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:09.901158 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:09.901260 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:09.901475 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:09.901477 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:09.901680 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:09.901780 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:09.901785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.902648 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.902652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:09.903231 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:09.903241 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.903691 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.903694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:09.903756 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:09.904295 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:09.904306 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:09.906380 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":125,"threads_started":1}
W20260810 18:54:09.906663 12499 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:09.900901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:38505 (local address 127.30.153.190:46099)
0810 18:54:09.901060 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:09.901064 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:09.901245 (+ 181us) server_negotiation.cc:407] Connection header received
0810 18:54:09.901334 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:09.901336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:09.901387 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:09.901463 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:09.901905 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.902515 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.903347 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:09.903568 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:09.903654 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:09.906550 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:38505: BlockingRecv error: recv got EOF from 127.30.153.130:38505 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 18:54:10 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:10 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:10.106309 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:10.100793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:51033)
0810 18:54:10.101082 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:10.101096 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:10.101234 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:10.101485 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:10.101487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:10.101685 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:10.101781 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:10.101786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.102580 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.102582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:10.103175 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:10.103184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.103667 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.103669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:10.103733 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:10.104201 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:10.104213 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:10.106189 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
W20260810 18:54:10.106479 12499 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:10.100919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:51033 (local address 127.30.153.190:46099)
0810 18:54:10.101194 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:10.101198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:10.101213 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:54:10.101322 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:10.101324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:10.101373 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:10.101454 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:10.101911 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.102489 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.103317 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.103541 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.103658 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:10.106333 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:51033: BlockingRecv error: recv got EOF from 127.30.153.131:51033 (error 108)
Metrics: {"server-negotiator.queue_time_us":189}
Aug 10 18:54:10 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:10 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:10.701330 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:10.695888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:37967)
0810 18:54:10.696198 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:10.696212 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:10.696309 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:10.696556 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:10.696559 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:10.696748 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:10.696849 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:10.696854 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.697587 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.697590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:10.698161 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:10.698169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.698679 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.698683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:10.698771 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:10.699340 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:10.699355 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:10.701213 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":99,"threads_started":1}
W20260810 18:54:10.701529 12506 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:10.695939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:37967 (local address 127.30.153.190:46099)
0810 18:54:10.696196 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:10.696200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:10.696277 (+ 77us) server_negotiation.cc:407] Connection header received
0810 18:54:10.696386 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:10.696388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:10.696434 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:10.696520 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:10.696945 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.697475 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.698267 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.698560 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.698668 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:10.701352 (+ 2684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:37967: BlockingRecv error: recv got EOF from 127.30.153.129:37967 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":95,"threads_started":1}
Aug 10 18:54:10 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:10 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:10.913302 12507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:10.907476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:53101)
0810 18:54:10.907756 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:10.907769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:10.907867 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:10.908145 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:10.908147 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:10.908331 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:10.908427 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:10.908433 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.909264 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.909267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:10.909831 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:10.909840 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.910326 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.910329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:10.910397 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:10.911003 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:10.911016 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:10.913096 (+ 2080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":106,"threads_started":1}
W20260810 18:54:10.913408 12506 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:10.907595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:53101 (local address 127.30.153.190:46099)
0810 18:54:10.907753 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:10.907757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:10.907863 (+ 106us) server_negotiation.cc:407] Connection header received
0810 18:54:10.907945 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:10.907947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:10.907996 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:10.908071 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:10.908568 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.909134 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.909950 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:10.910203 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:10.910290 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:10.913281 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:53101: BlockingRecv error: recv got EOF from 127.30.153.130:53101 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 18:54:11 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:11 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:11.113289 12512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.107267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:39047)
0810 18:54:11.107571 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.107590 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:11.107698 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:11.108020 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:11.108023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:11.108240 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:11.108348 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.108354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.109202 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.109205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.109765 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.109774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.110291 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.110294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.110366 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.111005 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:11.111027 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:11.113166 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":109,"threads_started":1}
W20260810 18:54:11.113543 12506 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:11.107428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:39047 (local address 127.30.153.190:46099)
0810 18:54:11.107594 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:11.107598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:11.107708 (+ 110us) server_negotiation.cc:407] Connection header received
0810 18:54:11.107783 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:11.107785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:11.107835 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:11.107902 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:11.108483 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.109033 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.109889 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.110158 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.110247 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.113326 (+ 3079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:39047: BlockingRecv error: recv got EOF from 127.30.153.131:39047 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 10 18:54:11 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:11 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:11.708395 12513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.702132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:52289)
0810 18:54:11.702461 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.702478 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:11.702588 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:11.702842 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:11.702844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:11.703059 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:11.703196 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.703203 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.704022 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.704025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.704669 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.704680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.705163 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.705165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.705236 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.705862 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:11.705876 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:11.708243 (+ 2367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":105,"threads_started":1}
W20260810 18:54:11.708624 12514 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:11.702285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:52289 (local address 127.30.153.190:46099)
0810 18:54:11.702565 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:11.702569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:11.702583 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:54:11.702665 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:11.702667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:11.702711 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:11.702796 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:11.703326 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.703890 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.704803 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.705042 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.705139 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.708463 (+ 3324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:52289: BlockingRecv error: recv got EOF from 127.30.153.129:52289 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":86,"threads_started":1}
Aug 10 18:54:11 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:11 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:11.920305 12515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:11.914213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:42761)
0810 18:54:11.914529 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:11.914544 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:11.914676 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:11.914971 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:11.914974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:11.915174 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:11.915276 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.915282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.916050 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.916053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.916600 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:11.916608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.917092 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.917094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:11.917156 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.917761 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:11.917778 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:11.920114 (+ 2336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":98,"threads_started":1}
W20260810 18:54:11.920378 12514 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:11.914257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:42761 (local address 127.30.153.190:46099)
0810 18:54:11.914477 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:11.914481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:11.914621 (+ 140us) server_negotiation.cc:407] Connection header received
0810 18:54:11.914763 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:11.914765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:11.914813 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:11.914887 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:11.915410 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.915950 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.916740 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:11.916987 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:11.917079 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:11.920258 (+ 3179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:42761: BlockingRecv error: recv got EOF from 127.30.153.130:42761 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 10 18:54:12 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:12 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:12.128559 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.114152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:60041)
0810 18:54:12.114589 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.114610 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:12.114752 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:12.115102 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:12.115106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:12.115320 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:12.115422 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.115427 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.116220 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.116225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.116954 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.116965 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.117774 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.117777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.117862 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.126216 (+ 8354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:12.126232 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:12.128371 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":120,"threads_started":1}
W20260810 18:54:12.129643 12514 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.114300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:60041 (local address 127.30.153.190:46099)
0810 18:54:12.114545 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.114551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.114698 (+ 147us) server_negotiation.cc:407] Connection header received
0810 18:54:12.114866 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.114872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.114928 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.115028 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.115560 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.116096 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.117093 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.117479 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.117599 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.128567 (+ 10968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:60041: BlockingRecv error: recv got EOF from 127.30.153.131:60041 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 10 18:54:12 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:12 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:12.715739 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.709349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:58607)
0810 18:54:12.709679 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.709694 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:12.709794 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:12.710161 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:12.710164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:12.710369 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:12.710555 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.710563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.711360 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.711363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.711930 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.711938 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.712520 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.712524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.712622 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.713324 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:12.713341 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:12.715599 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":111,"threads_started":1}
W20260810 18:54:12.715950 12522 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.709460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:58607 (local address 127.30.153.190:46099)
0810 18:54:12.709724 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.709727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.709745 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:54:12.709932 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.709935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.709988 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.710081 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.710687 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.711246 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.712069 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.712386 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.712471 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.715777 (+ 3306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:58607: BlockingRecv error: recv got EOF from 127.30.153.129:58607 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":102,"threads_started":1}
Aug 10 18:54:12 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:12 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:12.932749 12523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:12.921186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:57853)
0810 18:54:12.921478 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:12.921493 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:12.921604 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:12.922073 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:12.922075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:12.922277 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:12.922382 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.922388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.923265 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.923268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.924045 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:12.924066 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.924880 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.924886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:12.924989 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.930480 (+ 5491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:12.930492 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:12.932608 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":106,"threads_started":1}
W20260810 18:54:12.932960 12522 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:12.921283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:57853 (local address 127.30.153.190:46099)
0810 18:54:12.921420 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:12.921423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:12.921560 (+ 137us) server_negotiation.cc:407] Connection header received
0810 18:54:12.921718 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:12.921721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:12.921767 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:12.922007 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:12.922522 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.923087 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.924238 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:12.924564 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:12.924714 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:12.932812 (+ 8098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:57853: BlockingRecv error: recv got EOF from 127.30.153.130:57853 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 10 18:54:13 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:13 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:13.135769 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:13.129610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:56647)
0810 18:54:13.130037 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:13.130052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:13.130243 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:13.130550 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:13.130553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:13.130777 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:13.130868 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:13.130874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.131677 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.131684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:13.132260 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:13.132268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.132762 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.132765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:13.132856 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.133396 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:13.133409 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:13.135653 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":89,"threads_started":1}
W20260810 18:54:13.135910 12522 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:13.129741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:56647 (local address 127.30.153.190:46099)
0810 18:54:13.130003 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.130007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.130187 (+ 180us) server_negotiation.cc:407] Connection header received
0810 18:54:13.130352 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.130357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.130439 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.130529 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:13.130991 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.131524 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.132382 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.132619 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.132748 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.135788 (+ 3040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:56647: BlockingRecv error: recv got EOF from 127.30.153.131:56647 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Aug 10 18:54:13 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:13 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:13.722270 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:13.716561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:44971)
0810 18:54:13.716927 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:13.716942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:13.717052 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:13.717424 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:13.717427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:13.717626 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:13.717726 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:13.717732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.718537 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.718540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:13.719115 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:13.719123 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.719554 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.719556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:13.719619 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.720136 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:13.720149 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:13.722117 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":121,"threads_started":1}
W20260810 18:54:13.722565 12526 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:13.716702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:44971 (local address 127.30.153.190:46099)
0810 18:54:13.716965 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.716969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.716999 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:54:13.717198 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.717201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.717251 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.717363 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:13.717839 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.718370 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.719223 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.719452 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.719534 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.722368 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:44971: BlockingRecv error: recv got EOF from 127.30.153.129:44971 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":65,"threads_started":1}
Aug 10 18:54:13 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:13 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:13.939791 12527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:13.933695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:41235)
0810 18:54:13.934088 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:13.934105 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:13.934239 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:13.934564 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:13.934567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:13.934854 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:13.934980 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:13.934989 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.935985 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.935988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:13.936544 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:13.936553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.936985 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.936988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:13.937052 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.937627 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:13.937638 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:13.939672 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":142,"threads_started":1}
W20260810 18:54:13.940054 12526 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:13.933920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:41235 (local address 127.30.153.190:46099)
0810 18:54:13.934082 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:13.934086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:13.934212 (+ 126us) server_negotiation.cc:407] Connection header received
0810 18:54:13.934353 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:13.934357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:13.934443 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:13.934512 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:13.935171 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.935878 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.936666 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:13.936883 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:13.936967 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:13.939867 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:41235: BlockingRecv error: recv got EOF from 127.30.153.130:41235 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 18:54:14 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:14 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:14.142504 12529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:14.136544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:59645)
0810 18:54:14.136845 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:14.136861 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:14.136971 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:14.137217 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:14.137219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:14.137431 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:14.137531 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:14.137536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.138301 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.138303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:14.138997 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:14.139010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.139478 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.139481 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:14.139547 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:14.140046 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:14.140059 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:14.142339 (+ 2280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":72,"threads_started":1}
W20260810 18:54:14.142652 12526 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:14.136700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:59645 (local address 127.30.153.190:46099)
0810 18:54:14.136848 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:14.136851 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:14.136915 (+ 64us) server_negotiation.cc:407] Connection header received
0810 18:54:14.137056 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:14.137058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:14.137105 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:14.137173 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:14.137635 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.138156 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.139121 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.139375 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.139459 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:14.142532 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:59645: BlockingRecv error: recv got EOF from 127.30.153.131:59645 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 18:54:14 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:14 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:14.730762 12531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:14.723222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:60783)
0810 18:54:14.724928 (+ 1706us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:14.724942 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:14.725057 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:14.725323 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:14.725325 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:14.725528 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:14.725633 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:14.725639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.726503 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.726506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:14.727064 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:14.727071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.727567 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.727573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:14.727665 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:14.728234 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:14.728252 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:14.730643 (+ 2391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1610,"thread_start_us":1469,"threads_started":1}
W20260810 18:54:14.731011 12532 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:14.723396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:60783 (local address 127.30.153.190:46099)
0810 18:54:14.724911 (+ 1515us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:14.724915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:14.725071 (+ 156us) server_negotiation.cc:407] Connection header received
0810 18:54:14.725126 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:14.725128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:14.725178 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:14.725276 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:14.725756 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.726329 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.727170 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.727419 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.727502 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:14.730795 (+ 3293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:60783: BlockingRecv error: recv got EOF from 127.30.153.129:60783 (error 108)
Metrics: {"server-negotiator.queue_time_us":1435,"thread_start_us":1265,"threads_started":1}
Aug 10 18:54:14 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:14 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:14.950703 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:14.940845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:33113)
0810 18:54:14.941400 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:14.941423 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:14.941617 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:14.942267 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:14.942272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:14.942591 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:14.942755 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:14.942767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.943970 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.943975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:14.944979 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:14.944991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.945721 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.945728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:14.945870 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:14.946571 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:14.946586 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:14.950513 (+ 3927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":261,"threads_started":1}
W20260810 18:54:14.950961 12532 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:14.941198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:33113 (local address 127.30.153.190:46099)
0810 18:54:14.941389 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:14.941396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:14.941510 (+ 114us) server_negotiation.cc:407] Connection header received
0810 18:54:14.941768 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:14.941773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:14.941842 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:14.941901 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:14.942922 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.943855 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.945153 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:14.945577 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:14.945729 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:14.950791 (+ 5062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:33113: BlockingRecv error: recv got EOF from 127.30.153.130:33113 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 18:54:15 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:15 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.131@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:15.149520 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:15.143498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.131:58357)
0810 18:54:15.143814 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:15.143829 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:15.143936 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:15.144197 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:15.144200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:15.144409 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:15.144514 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:15.144520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.145284 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.145287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:15.145913 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:15.145922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.146484 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.146492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:15.146575 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:15.147086 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:15.147098 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:15.149340 (+ 2242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":129,"threads_started":1}
W20260810 18:54:15.149587 12532 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:15.143613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:58357 (local address 127.30.153.190:46099)
0810 18:54:15.143759 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:15.143763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:15.143902 (+ 139us) server_negotiation.cc:407] Connection header received
0810 18:54:15.144008 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:15.144010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:15.144059 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:15.144131 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:15.144628 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.145165 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.146044 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.146290 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.146375 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:15.149473 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.131:58357: BlockingRecv error: recv got EOF from 127.30.153.131:58357 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 18:54:15 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:15 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.129@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:15.737584 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:15.731802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.129:50839)
0810 18:54:15.732149 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:15.732164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:15.732254 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:15.732526 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:15.732528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:15.732729 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:15.732829 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:15.732834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.733586 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.733589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:15.734272 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:15.734282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.734880 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.734885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:15.734985 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:15.735593 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:15.735608 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:15.737455 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":100,"threads_started":1}
W20260810 18:54:15.737756 12541 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:15.731949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:50839 (local address 127.30.153.190:46099)
0810 18:54:15.732196 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:15.732200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:15.732215 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:54:15.732353 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:15.732356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:15.732400 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:15.732480 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:15.732941 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.733485 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.734382 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.734748 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.734864 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:15.737599 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.129:50839: BlockingRecv error: recv got EOF from 127.30.153.129:50839 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":64,"threads_started":1}
Aug 10 18:54:15 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 18:54:15 dist-test-slave-68xk krb5kdc[10946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.153.130@KRBTEST.COM for oryx/127.30.153.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 18:54:15.958559 12542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:15.951972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:46099 (local address 127.30.153.130:59893)
0810 18:54:15.952436 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:15.952453 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:15.952602 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:15.952894 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:15.952897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:15.953123 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:15.953215 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:15.953221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.954199 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.954206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:15.954928 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:15.954938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.955417 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.955421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:15.955510 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:15.956122 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:15.956135 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:15.958326 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.153.190:46099: Server oryx/127.30.153.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":90,"threads_started":1}
W20260810 18:54:15.958623 12541 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:15.952091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:59893 (local address 127.30.153.190:46099)
0810 18:54:15.952436 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:15.952442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:15.952539 (+ 97us) server_negotiation.cc:407] Connection header received
0810 18:54:15.952721 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:15.952724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:15.952780 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:15.952869 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:15.953333 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.954044 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.955054 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:15.955292 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:15.955401 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:15.958502 (+ 3101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.153.130:59893: BlockingRecv error: recv got EOF from 127.30.153.130:59893 (error 108)
Metrics: {"server-negotiator.queue_time_us":230}
I20260810 18:54:16.040714 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 11808
I20260810 18:54:16.052875 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 12044
I20260810 18:54:16.068866 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 12247
I20260810 18:54:16.080380 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 11704
2026-08-10T18:54:16Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18433 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T18:54:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:16Z Disabled control of system clock
I20260810 18:54:16.122141 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42125
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:39183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:42125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 18:54:16.236869 12548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:16.237180 12548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:16.237262 12548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:16.241215 12548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:16.241313 12548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:16.241344 12548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:16.241385 12548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:16.241421 12548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:16.246304 12548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:42125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:16.247403 12548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:16.248838 12548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:16.258452 12553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.258460 12559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:16.258514 12548 server_base.cc:1034] running on GCE node
W20260810 18:54:16.258560 12554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:16.259402 12548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:16.260519 12548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:16.261688 12548 hybrid_clock.cc:648] HybridClock initialized: now 1786388056261674 us; error 27 us; skew 500 ppm
I20260810 18:54:16.263716 12548 webserver.cc:460] Webserver started at http://127.30.153.190:34551/ using document root <none> and password file <none>
I20260810 18:54:16.264371 12548 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:16.264441 12548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:16.264696 12548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:16.266881 12548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "6ee1089fdcdc42249ea8edb95a09645b"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "f6b6a7209df96015863571250db925b5"
server_key_iv: "12dcece16d6bc9241ab50a5da60f52c4"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.267468 12548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "6ee1089fdcdc42249ea8edb95a09645b"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "f6b6a7209df96015863571250db925b5"
server_key_iv: "12dcece16d6bc9241ab50a5da60f52c4"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.271622 12548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:54:16.274073 12565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.275427 12548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:54:16.275574 12548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "6ee1089fdcdc42249ea8edb95a09645b"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "f6b6a7209df96015863571250db925b5"
server_key_iv: "12dcece16d6bc9241ab50a5da60f52c4"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.275692 12548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:16.290892 12548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:16.291587 12548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:16.291795 12548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:16.299355 12548 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:42125
I20260810 18:54:16.299372 12650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:42125 every 8 connection(s)
I20260810 18:54:16.300321 12548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:16.303792 12651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:16.309608 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 12548
I20260810 18:54:16.309723 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:54:16.309870 12651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b: Bootstrap starting.
I20260810 18:54:16.310130 31334 external_mini_cluster.cc:1428] Setting key dc9c8d0ab7d34a3fac1f5b0f27930f9f
I20260810 18:54:16.312788 12651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:16.313603 12651 log.cc:824] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:16.315964 12651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b: No bootstrap required, opened a new log
I20260810 18:54:16.320155 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.311674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.30.153.190:42125)
0810 18:54:16.312379 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.312407 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.312445 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:54:16.313134 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.313153 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.313558 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.313893 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.314342 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.315380 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.316760 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.317220 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.317351 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.318329 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:16.318334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:16.318346 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:16.318790 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:16.318846 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:16.318867 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:16.318946 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:16.319126 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 18:54:16.319440 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":140,"threads_started":1}
I20260810 18:54:16.320693 12651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee1089fdcdc42249ea8edb95a09645b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42125 } }
I20260810 18:54:16.320973 12651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:16.321036 12651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee1089fdcdc42249ea8edb95a09645b, State: Initialized, Role: FOLLOWER
I20260810 18:54:16.321513 12651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee1089fdcdc42249ea8edb95a09645b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42125 } }
I20260810 18:54:16.321646 12651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:16.321744 12651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:16.321883 12651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:16.323055 12651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee1089fdcdc42249ea8edb95a09645b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42125 } }
I20260810 18:54:16.323419 12651 leader_election.cc:302] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ee1089fdcdc42249ea8edb95a09645b; no voters:
I20260810 18:54:16.323765 12651 leader_election.cc:288] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:54:16.323901 12656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:16.324177 12656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [term 1 LEADER]: Becoming Leader. State: Replica: 6ee1089fdcdc42249ea8edb95a09645b, State: Running, Role: LEADER
I20260810 18:54:16.324566 12656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee1089fdcdc42249ea8edb95a09645b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42125 } }
I20260810 18:54:16.325080 12651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:16.326284 12657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ee1089fdcdc42249ea8edb95a09645b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee1089fdcdc42249ea8edb95a09645b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42125 } } }
I20260810 18:54:16.326361 12659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee1089fdcdc42249ea8edb95a09645b. Latest consensus state: current_term: 1 leader_uuid: "6ee1089fdcdc42249ea8edb95a09645b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee1089fdcdc42249ea8edb95a09645b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42125 } } }
I20260810 18:54:16.326502 12659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:16.326444 12657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b [sys.catalog]: This master's current role is: LEADER
W20260810 18:54:16.330564 12677 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:54:16.330659 12677 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:54:16.330811 12667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:16.331740 12667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:16.337991 12667 catalog_manager.cc:1347] Generated new cluster ID: e9e816219d2749ef943b2c4cb89bcd44
I20260810 18:54:16.338073 12667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:16.347611 12667 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:16.348538 12667 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:16.354516 12667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ee1089fdcdc42249ea8edb95a09645b: Generated new TSK 0
I20260810 18:54:16.355191 12667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:16.359858 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42125
--builtin_ntp_servers=127.30.153.148:39183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 18:54:16.485719 12680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:16.486014 12680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:16.486088 12680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:16.490240 12680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:16.490298 12680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:16.490386 12680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:54:16.495134 12680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:16.496223 12680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:16.497728 12680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:16.507725 12692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:16.507814 12680 server_base.cc:1034] running on GCE node
W20260810 18:54:16.507741 12694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.507750 12697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:16.508400 12680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:16.509065 12680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:16.510265 12680 hybrid_clock.cc:648] HybridClock initialized: now 1786388056510248 us; error 28 us; skew 500 ppm
I20260810 18:54:16.512257 12680 webserver.cc:460] Webserver started at http://127.30.153.129:36987/ using document root <none> and password file <none>
I20260810 18:54:16.512874 12680 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:16.512976 12680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:16.513222 12680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:16.515372 12680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "072fa00ae9a34ee3ab129acdf8c8126d"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "bb87b6a4ed6b737628f6d32f0c48adce"
server_key_iv: "69322feaed03a01aeb218a1602a22bbb"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.515956 12680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "072fa00ae9a34ee3ab129acdf8c8126d"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "bb87b6a4ed6b737628f6d32f0c48adce"
server_key_iv: "69322feaed03a01aeb218a1602a22bbb"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.519876 12680 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:54:16.522405 12707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.523708 12680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:54:16.523869 12680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "072fa00ae9a34ee3ab129acdf8c8126d"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "bb87b6a4ed6b737628f6d32f0c48adce"
server_key_iv: "69322feaed03a01aeb218a1602a22bbb"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.523990 12680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:16.539283 12680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:16.539971 12680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:16.540175 12680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:16.540774 12680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:16.541729 12680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:16.541811 12680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.541884 12680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:16.541937 12680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.551656 12680 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:40751
I20260810 18:54:16.551680 12889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:40751 every 8 connection(s)
I20260810 18:54:16.552564 12680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:54:16.558477 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 12680
I20260810 18:54:16.558620 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:54:16.558928 31334 external_mini_cluster.cc:1428] Setting key 91ad9c8ec741595c02dcf905266287e4
I20260810 18:54:16.561010 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42125
--builtin_ntp_servers=127.30.153.148:39183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 18:54:16.563129 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.554614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:44579 (local address 127.30.153.190:42125)
0810 18:54:16.554788 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.554793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.555936 (+ 1143us) server_negotiation.cc:407] Connection header received
0810 18:54:16.557435 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.557438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.557489 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.557579 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.558664 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.559359 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.560876 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.561108 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.561272 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.562566 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:16.562575 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:16.562578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:16.562590 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:16.562611 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:16.562621 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:16.562714 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:16.562906 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 18:54:16.562969 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:54:16.563992 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.555052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42125 (local address 127.30.153.129:44579)
0810 18:54:16.555769 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.555816 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:16.557192 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:16.557754 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:16.557760 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:16.557847 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:16.558391 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.558406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.559550 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.559553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.560727 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.560740 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.561327 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.561332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.561518 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.562213 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:54:16.562215 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:16.562257 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:54:16.562322 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:16.562719 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:16.562732 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:16.562741 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:54:16.562883 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 18:54:16.563175 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":120,"threads_started":1}
I20260810 18:54:16.565353 12891 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42125
I20260810 18:54:16.565562 12891 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:16.566092 12891 heartbeater.cc:511] Master 127.30.153.190:42125 requested a full tablet report, sending...
I20260810 18:54:16.567728 12597 ts_manager.cc:194] Registered new tserver with Master: 072fa00ae9a34ee3ab129acdf8c8126d (127.30.153.129:40751)
I20260810 18:54:16.568984 12597 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.129:44579
W20260810 18:54:16.676087 12899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:16.676368 12899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:16.676432 12899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:16.680632 12899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:16.680706 12899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:16.680794 12899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:54:16.685504 12899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:16.686617 12899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:16.688094 12899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:16.698017 12908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.697989 12907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.698001 12913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:16.698480 12899 server_base.cc:1034] running on GCE node
I20260810 18:54:16.698987 12899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:16.699689 12899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:16.700902 12899 hybrid_clock.cc:648] HybridClock initialized: now 1786388056700867 us; error 49 us; skew 500 ppm
I20260810 18:54:16.703047 12899 webserver.cc:460] Webserver started at http://127.30.153.130:46023/ using document root <none> and password file <none>
I20260810 18:54:16.703720 12899 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:16.703826 12899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:16.704101 12899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:16.706537 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "80f4fecaf9eb49f2a8337a6cffe86dde"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "ae6758710bc342051bae3ffe7531db73"
server_key_iv: "6164d6b682111910ab62987a416cfa78"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.707248 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "80f4fecaf9eb49f2a8337a6cffe86dde"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "ae6758710bc342051bae3ffe7531db73"
server_key_iv: "6164d6b682111910ab62987a416cfa78"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.711232 12899 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:54:16.713995 12922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.715371 12899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:54:16.715517 12899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "80f4fecaf9eb49f2a8337a6cffe86dde"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "ae6758710bc342051bae3ffe7531db73"
server_key_iv: "6164d6b682111910ab62987a416cfa78"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.715644 12899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:16.728125 12899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:16.728827 12899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:16.729034 12899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:16.729632 12899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:16.730656 12899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:16.730731 12899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.730805 12899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:16.730850 12899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.740186 12899 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:34691
I20260810 18:54:16.740231 13079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:34691 every 8 connection(s)
I20260810 18:54:16.741228 12899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:54:16.748733 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 12899
I20260810 18:54:16.748858 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:54:16.749183 31334 external_mini_cluster.cc:1428] Setting key 844d725b21e9682f318415d45f1bf159
I20260810 18:54:16.751271 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42125
--builtin_ntp_servers=127.30.153.148:39183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 18:54:16.751727 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.743163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:54475 (local address 127.30.153.190:42125)
0810 18:54:16.743332 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.743337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.744552 (+ 1215us) server_negotiation.cc:407] Connection header received
0810 18:54:16.746063 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.746067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.746117 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.746202 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.747225 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.747885 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.749432 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.749735 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.749852 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.751170 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:16.751174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:16.751177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:16.751194 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:16.751213 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:16.751221 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:16.751298 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:16.751515 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 18:54:16.751583 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:54:16.752606 13088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.743559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42125 (local address 127.30.153.130:54475)
0810 18:54:16.744381 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.744434 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:16.745860 (+ 1426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:16.746363 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:16.746369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:16.746545 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:16.747042 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.747056 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.748027 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.748030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.749300 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.749313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.749918 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.749922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.750115 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.750873 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:54:16.750875 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:16.750915 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:54:16.750981 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:16.751314 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:16.751327 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:16.751335 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 18:54:16.751508 (+ 173us) client_negotiation.cc:240] Negotiation successful
0810 18:54:16.751800 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":180,"threads_started":1}
I20260810 18:54:16.754168 13080 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42125
I20260810 18:54:16.754472 13080 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:16.755014 13080 heartbeater.cc:511] Master 127.30.153.190:42125 requested a full tablet report, sending...
I20260810 18:54:16.756158 12597 ts_manager.cc:194] Registered new tserver with Master: 80f4fecaf9eb49f2a8337a6cffe86dde (127.30.153.130:34691)
I20260810 18:54:16.756749 12597 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.130:54475
W20260810 18:54:16.866999 13089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:16.867313 13089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:16.867393 13089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:16.871790 13089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:16.871872 13089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:16.871960 13089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:54:16.876782 13089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:16.877986 13089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:16.879547 13089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:16.889498 13102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.889501 13099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:16.889472 13100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:16.889912 13089 server_base.cc:1034] running on GCE node
I20260810 18:54:16.890358 13089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:16.891073 13089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:16.892256 13089 hybrid_clock.cc:648] HybridClock initialized: now 1786388056892239 us; error 33 us; skew 500 ppm
I20260810 18:54:16.894397 13089 webserver.cc:460] Webserver started at http://127.30.153.131:36259/ using document root <none> and password file <none>
I20260810 18:54:16.895048 13089 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:16.895121 13089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:16.895346 13089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:16.897460 13089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "d027590b16754da7b484c824273f4e8a"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "97939179e178b2f52c679f77980ee32f"
server_key_iv: "5d608e9b034703ce0869a9bd298841cc"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.898031 13089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "d027590b16754da7b484c824273f4e8a"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "97939179e178b2f52c679f77980ee32f"
server_key_iv: "5d608e9b034703ce0869a9bd298841cc"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.901814 13089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.003s
I20260810 18:54:16.904692 13109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.906038 13089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:54:16.906219 13089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "d027590b16754da7b484c824273f4e8a"
format_stamp: "Formatted at 2026-08-10 18:54:16 on dist-test-slave-68xk"
server_key: "97939179e178b2f52c679f77980ee32f"
server_key_iv: "5d608e9b034703ce0869a9bd298841cc"
server_key_version: "encryptionkey@0"
I20260810 18:54:16.906332 13089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:16.919874 13089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:16.920616 13089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:16.920809 13089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:16.921553 13089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:16.922639 13089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:16.922693 13089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.922734 13089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:16.922749 13089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:16.931910 13089 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:40329
I20260810 18:54:16.932009 13284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:40329 every 8 connection(s)
I20260810 18:54:16.932881 13089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:54:16.939150 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 13089
I20260810 18:54:16.939275 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:54:16.939575 31334 external_mini_cluster.cc:1428] Setting key bdb9bb53cb5298df064db55db224c905
I20260810 18:54:16.943363 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.934928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:33217 (local address 127.30.153.190:42125)
0810 18:54:16.935095 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.935100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.936275 (+ 1175us) server_negotiation.cc:407] Connection header received
0810 18:54:16.937800 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.937802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.937851 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.937929 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.939019 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.939849 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.941288 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.941594 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.941703 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.942893 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:16.942896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:16.942898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:16.942910 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:16.942931 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:16.942946 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:16.943018 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:16.943194 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 18:54:16.943252 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:54:16.944171 13287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:16.935364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42125 (local address 127.30.153.131:33217)
0810 18:54:16.936106 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:16.936157 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:16.937579 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:16.938117 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:16.938124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:16.938215 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:16.938859 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.938873 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.940007 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.940011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.941159 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:16.941171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:16.941779 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:16.941784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:16.941968 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:16.942601 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:54:16.942603 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:16.942642 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:54:16.942727 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:16.943031 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:16.943038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:16.943042 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:54:16.943135 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:54:16.943351 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":115,"threads_started":1}
I20260810 18:54:16.945322 13285 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42125
I20260810 18:54:16.945581 13285 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:16.946064 13285 heartbeater.cc:511] Master 127.30.153.190:42125 requested a full tablet report, sending...
I20260810 18:54:16.947165 12597 ts_manager.cc:194] Registered new tserver with Master: d027590b16754da7b484c824273f4e8a (127.30.153.131:40329)
I20260810 18:54:16.947779 12597 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.131:33217
I20260810 18:54:16.954092 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:54:16.957219 12654 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:16.956176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.30.153.190:42125)
0810 18:54:16.956343 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:16.956348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:16.956498 (+ 150us) server_negotiation.cc:407] Connection header received
0810 18:54:16.956657 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:16.956665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:16.956708 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:16.956794 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:16.957089 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34518: BlockingRecv error: recv got EOF from 127.0.0.1:34518 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:54:16.957408 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 12680
I20260810 18:54:16.980106 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 12899
I20260810 18:54:16.995461 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 13089
I20260810 18:54:17.014149 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 12548
2026-08-10T18:54:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (927 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T18:54:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:17Z Disabled control of system clock
I20260810 18:54:17.049023 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42279
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:42279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:54:17.162624 13303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:17.162889 13303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:17.162951 13303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:17.167111 13303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:17.167171 13303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:17.167186 13303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:17.167203 13303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:17.167218 13303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:17.171981 13303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:42279
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:17.173084 13303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:17.174600 13303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:17.183362 13311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.183368 13312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.183377 13314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:17.183728 13303 server_base.cc:1034] running on GCE node
I20260810 18:54:17.184388 13303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:17.185578 13303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:17.186782 13303 hybrid_clock.cc:648] HybridClock initialized: now 1786388057186750 us; error 46 us; skew 500 ppm
I20260810 18:54:17.188980 13303 webserver.cc:460] Webserver started at http://127.30.153.190:33785/ using document root <none> and password file <none>
I20260810 18:54:17.189635 13303 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:17.189749 13303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:17.190013 13303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:17.192451 13303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "0a775b913f784429aeeb24245c1441c3"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "9d052006eadab8760cde3c0733645798"
server_key_iv: "7df0bb99287bf60ba4e103d92cff58c2"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.193145 13303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "0a775b913f784429aeeb24245c1441c3"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "9d052006eadab8760cde3c0733645798"
server_key_iv: "7df0bb99287bf60ba4e103d92cff58c2"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.197497 13303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:54:17.200240 13326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.201586 13303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:54:17.201781 13303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "0a775b913f784429aeeb24245c1441c3"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "9d052006eadab8760cde3c0733645798"
server_key_iv: "7df0bb99287bf60ba4e103d92cff58c2"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.201908 13303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:17.206691 13303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:17.207326 13303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:17.207528 13303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:17.215088 13303 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:42279
I20260810 18:54:17.215109 13411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:42279 every 8 connection(s)
I20260810 18:54:17.216097 13303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:17.219584 13413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:17.225198 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 13303
I20260810 18:54:17.225309 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:54:17.225376 13413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3: Bootstrap starting.
I20260810 18:54:17.225636 31334 external_mini_cluster.cc:1428] Setting key b72f0a2cc0f0925c26f4162d194e7db2
I20260810 18:54:17.228106 13413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:17.228884 13413 log.cc:824] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:17.231304 13413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3: No bootstrap required, opened a new log
I20260810 18:54:17.232000 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.227148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.30.153.190:42279)
0810 18:54:17.227834 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.227850 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.227887 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:54:17.228598 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.228617 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.228987 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.229378 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.230208 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.230214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.230229 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:17.230240 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.230308 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:17.230331 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.230485 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.230685 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.231035 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":145,"threads_started":1}
I20260810 18:54:17.235085 13413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a775b913f784429aeeb24245c1441c3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42279 } }
I20260810 18:54:17.235316 13413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:17.235383 13413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a775b913f784429aeeb24245c1441c3, State: Initialized, Role: FOLLOWER
I20260810 18:54:17.235791 13413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a775b913f784429aeeb24245c1441c3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42279 } }
I20260810 18:54:17.235914 13413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:17.235998 13413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:17.236114 13413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:17.237049 13413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a775b913f784429aeeb24245c1441c3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42279 } }
I20260810 18:54:17.237397 13413 leader_election.cc:302] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a775b913f784429aeeb24245c1441c3; no voters:
I20260810 18:54:17.237751 13413 leader_election.cc:288] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:54:17.237892 13421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:17.238147 13421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [term 1 LEADER]: Becoming Leader. State: Replica: 0a775b913f784429aeeb24245c1441c3, State: Running, Role: LEADER
I20260810 18:54:17.238509 13421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a775b913f784429aeeb24245c1441c3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42279 } }
I20260810 18:54:17.239114 13413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:17.240322 13422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a775b913f784429aeeb24245c1441c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a775b913f784429aeeb24245c1441c3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42279 } } }
I20260810 18:54:17.240334 13424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a775b913f784429aeeb24245c1441c3. Latest consensus state: current_term: 1 leader_uuid: "0a775b913f784429aeeb24245c1441c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a775b913f784429aeeb24245c1441c3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42279 } } }
I20260810 18:54:17.240471 13422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:17.240471 13424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:17.241008 13436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:17.244132 13436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:17.250162 13436 catalog_manager.cc:1347] Generated new cluster ID: 58009fb6a8d84862b4dbd5fbfaf7fbbb
I20260810 18:54:17.250293 13436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:17.260878 13436 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:17.261253 13436 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:17.281319 13436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a775b913f784429aeeb24245c1441c3: Generated new TSK 0
I20260810 18:54:17.282130 13436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:54:17.285102 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42279
--builtin_ntp_servers=127.30.153.148:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:54:17.418100 13448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:17.418363 13448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:17.418403 13448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:17.422408 13448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:17.422503 13448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:17.422619 13448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:54:17.427383 13448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:17.428483 13448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:17.429883 13448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:17.439390 13453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.439390 13454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.439445 13456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:17.439778 13448 server_base.cc:1034] running on GCE node
I20260810 18:54:17.440203 13448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:17.440903 13448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:17.442099 13448 hybrid_clock.cc:648] HybridClock initialized: now 1786388057442085 us; error 33 us; skew 500 ppm
I20260810 18:54:17.444028 13448 webserver.cc:460] Webserver started at http://127.30.153.129:45703/ using document root <none> and password file <none>
I20260810 18:54:17.444615 13448 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:17.444722 13448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:17.444960 13448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:17.447127 13448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "e2fdbc9e5eb64affb6d470f02cffd011"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "a097620b143351e71fdbd4c63c79bdcf"
server_key_iv: "09bfc1d38b444c51552fef8bc0b9a90b"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.447710 13448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "e2fdbc9e5eb64affb6d470f02cffd011"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "a097620b143351e71fdbd4c63c79bdcf"
server_key_iv: "09bfc1d38b444c51552fef8bc0b9a90b"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.451676 13448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:54:17.454519 13464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.455791 13448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:54:17.455946 13448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "e2fdbc9e5eb64affb6d470f02cffd011"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "a097620b143351e71fdbd4c63c79bdcf"
server_key_iv: "09bfc1d38b444c51552fef8bc0b9a90b"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.456081 13448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:17.461095 13448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:17.461747 13448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:17.461948 13448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:17.462575 13448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:17.463615 13448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:17.463691 13448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.463763 13448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:17.463819 13448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.473739 13448 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:40575
I20260810 18:54:17.473790 13651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:40575 every 8 connection(s)
I20260810 18:54:17.474776 13448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:54:17.481083 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.476660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:60791 (local address 127.30.153.190:42279)
0810 18:54:17.476825 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.476830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.477951 (+ 1121us) server_negotiation.cc:407] Connection header received
0810 18:54:17.479424 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.479427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.479468 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.479564 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.480683 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.480686 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.480688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:17.480700 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.480722 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:17.480730 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.480815 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.480937 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.480976 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:54:17.481920 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.477105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42279 (local address 127.30.153.129:60791)
0810 18:54:17.477797 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.477840 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:17.479179 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:17.479720 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:17.479727 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:17.479816 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:17.480408 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:54:17.480409 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:17.480449 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:54:17.480516 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:17.480812 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:17.480819 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:17.480823 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:54:17.480954 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 18:54:17.481151 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":119,"threads_started":1}
I20260810 18:54:17.483191 13652 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42279
I20260810 18:54:17.483431 13652 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:17.483654 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 13448
I20260810 18:54:17.483771 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:54:17.483925 13652 heartbeater.cc:511] Master 127.30.153.190:42279 requested a full tablet report, sending...
I20260810 18:54:17.484122 31334 external_mini_cluster.cc:1428] Setting key 8abd48213e197bcd35f1feec165397e5
I20260810 18:54:17.485561 13349 ts_manager.cc:194] Registered new tserver with Master: e2fdbc9e5eb64affb6d470f02cffd011 (127.30.153.129:40575)
I20260810 18:54:17.486196 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42279
--builtin_ntp_servers=127.30.153.148:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 18:54:17.597960 13661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:17.598284 13661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:17.598359 13661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:17.602394 13661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:17.602496 13661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:17.602587 13661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:54:17.608096 13661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:17.609206 13661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:17.610669 13661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:17.620405 13668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.620421 13672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.620422 13669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:17.620851 13661 server_base.cc:1034] running on GCE node
I20260810 18:54:17.621259 13661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:17.621917 13661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:17.623107 13661 hybrid_clock.cc:648] HybridClock initialized: now 1786388057623100 us; error 32 us; skew 500 ppm
I20260810 18:54:17.625083 13661 webserver.cc:460] Webserver started at http://127.30.153.130:32769/ using document root <none> and password file <none>
I20260810 18:54:17.625676 13661 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:17.625782 13661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:17.626024 13661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:17.628160 13661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "b776be3cae304b808ba28c88e989427c"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "7bd40935ca5f2577f326dcad5667e62b"
server_key_iv: "13aa46f0241b8b42c860cfcd79c84c1d"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.628755 13661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "b776be3cae304b808ba28c88e989427c"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "7bd40935ca5f2577f326dcad5667e62b"
server_key_iv: "13aa46f0241b8b42c860cfcd79c84c1d"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.632504 13661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260810 18:54:17.635341 13684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.636750 13661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:54:17.636898 13661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "b776be3cae304b808ba28c88e989427c"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "7bd40935ca5f2577f326dcad5667e62b"
server_key_iv: "13aa46f0241b8b42c860cfcd79c84c1d"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.637023 13661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:17.642328 13661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:17.642992 13661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:17.643194 13661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:17.643770 13661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:17.644881 13661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:17.644965 13661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.645054 13661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:17.645123 13661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.654901 13661 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:42399
I20260810 18:54:17.654938 13861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:42399 every 8 connection(s)
I20260810 18:54:17.655874 13661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:54:17.662587 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.657837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:52799 (local address 127.30.153.190:42279)
0810 18:54:17.658000 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.658005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.659181 (+ 1176us) server_negotiation.cc:407] Connection header received
0810 18:54:17.660772 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.660774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.660812 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.660892 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.662041 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.662044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.662046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:17.662057 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.662077 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:17.662085 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.662195 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.662372 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.662467 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:54:17.663237 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 13661
I20260810 18:54:17.663386 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:54:17.663652 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.658289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42279 (local address 127.30.153.130:52799)
0810 18:54:17.658997 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.659041 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:17.660551 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:17.661062 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:17.661101 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:17.661197 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:17.661785 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:54:17.661786 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:17.661827 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:54:17.661896 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:17.662214 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:17.662224 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:17.662231 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:54:17.662401 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 18:54:17.662730 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":142,"threads_started":1}
I20260810 18:54:17.663708 31334 external_mini_cluster.cc:1428] Setting key 51fe231fe0750f5dd90cf6877c4dcc01
I20260810 18:54:17.665083 13864 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42279
I20260810 18:54:17.665393 13864 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:17.665827 13864 heartbeater.cc:511] Master 127.30.153.190:42279 requested a full tablet report, sending...
I20260810 18:54:17.666018 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42279
--builtin_ntp_servers=127.30.153.148:36571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 18:54:17.667065 13354 ts_manager.cc:194] Registered new tserver with Master: b776be3cae304b808ba28c88e989427c (127.30.153.130:42399)
W20260810 18:54:17.783044 13867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:17.783316 13867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:17.783383 13867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:17.787362 13867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:17.787437 13867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:17.787523 13867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:54:17.792332 13867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:17.793386 13867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:17.794860 13867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:17.804875 13880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.804843 13876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:17.804868 13875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:17.805006 13867 server_base.cc:1034] running on GCE node
I20260810 18:54:17.805769 13867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:17.806578 13867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:17.807789 13867 hybrid_clock.cc:648] HybridClock initialized: now 1786388057807759 us; error 54 us; skew 500 ppm
I20260810 18:54:17.809825 13867 webserver.cc:460] Webserver started at http://127.30.153.131:35597/ using document root <none> and password file <none>
I20260810 18:54:17.810400 13867 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:17.810536 13867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:17.810732 13867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:17.812861 13867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "54ebb1bf8a964ce886d9bd01b2d3ba11"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "60cf4e2861c0f031488b1c43f8bff56d"
server_key_iv: "8c00bffd5e0e8ef0bd73b457962213f6"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.813400 13867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "54ebb1bf8a964ce886d9bd01b2d3ba11"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "60cf4e2861c0f031488b1c43f8bff56d"
server_key_iv: "8c00bffd5e0e8ef0bd73b457962213f6"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.817521 13867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 18:54:17.820254 13888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.821509 13867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:54:17.821662 13867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "54ebb1bf8a964ce886d9bd01b2d3ba11"
format_stamp: "Formatted at 2026-08-10 18:54:17 on dist-test-slave-68xk"
server_key: "60cf4e2861c0f031488b1c43f8bff56d"
server_key_iv: "8c00bffd5e0e8ef0bd73b457962213f6"
server_key_version: "encryptionkey@0"
I20260810 18:54:17.821782 13867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:17.826797 13867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:17.827487 13867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:17.827685 13867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:17.828258 13867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:17.829236 13867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:17.829286 13867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.829355 13867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:17.829394 13867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:17.839550 13867 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:37577
I20260810 18:54:17.839622 14069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:37577 every 8 connection(s)
I20260810 18:54:17.840699 13867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:54:17.843954 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 13867
I20260810 18:54:17.844074 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:54:17.844355 31334 external_mini_cluster.cc:1428] Setting key 4ae564024beada1b62a13669d295df47
I20260810 18:54:17.847538 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.842784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:43711 (local address 127.30.153.190:42279)
0810 18:54:17.842941 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.842945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.844036 (+ 1091us) server_negotiation.cc:407] Connection header received
0810 18:54:17.845671 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.845674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.845722 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.845814 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.847000 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:17.847004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:17.847007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:54:17.847021 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:17.847053 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:54:17.847072 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:17.847181 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:17.847346 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 18:54:17.847396 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:54:17.848534 14073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:17.843156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42279 (local address 127.30.153.131:43711)
0810 18:54:17.843893 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:17.843937 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:17.845416 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:17.845994 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:17.846002 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:17.846099 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:17.846723 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:54:17.846725 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:17.846768 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:54:17.846842 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:17.847181 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:17.847194 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:17.847204 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 18:54:17.847396 (+ 192us) client_negotiation.cc:240] Negotiation successful
0810 18:54:17.847675 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":105,"threads_started":1}
I20260810 18:54:17.849684 14070 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42279
I20260810 18:54:17.849947 14070 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:17.850356 14070 heartbeater.cc:511] Master 127.30.153.190:42279 requested a full tablet report, sending...
I20260810 18:54:17.851446 13354 ts_manager.cc:194] Registered new tserver with Master: 54ebb1bf8a964ce886d9bd01b2d3ba11 (127.30.153.131:37577)
I20260810 18:54:17.858484 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 18:54:17.861662 13417 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:54:17.860556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.30.153.190:42279)
0810 18:54:17.860700 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:17.860705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:17.860975 (+ 270us) server_negotiation.cc:407] Connection header received
0810 18:54:17.861082 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:17.861084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:17.861116 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:17.861183 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:17.861509 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34702: BlockingRecv error: recv got EOF from 127.0.0.1:34702 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:54:17.861822 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 13448
I20260810 18:54:17.882684 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 13661
I20260810 18:54:17.896617 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 13867
I20260810 18:54:17.914711 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 13303
2026-08-10T18:54:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (902 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:54:17 dist-test-slave-68xk krb5kdc[14082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:54:17 dist-test-slave-68xk krb5kdc[14082](info): set up 2 sockets
Aug 10 18:54:17 dist-test-slave-68xk krb5kdc[14082](info): commencing operation
krb5kdc: starting...
W20260810 18:54:19.180466 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:54:19 dist-test-slave-68xk krb5kdc[14082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:54:19.313685 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38493
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:19.430933 14100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:19.431229 14100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:19.431303 14100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:19.435209 14100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:54:19.435289 14100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:19.435328 14100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:19.435390 14100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:54:19.435431 14100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:54:19.440521 14100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:19.441536 14100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:19.442853 14100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:19.452099 14111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:19.452167 14108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:19.452131 14109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:19.452800 14100 server_base.cc:1034] running on GCE node
I20260810 18:54:19.453385 14100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:19.454659 14100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:19.455837 14100 hybrid_clock.cc:648] HybridClock initialized: now 1786388059455809 us; error 45 us; skew 500 ppm
Aug 10 18:54:19 dist-test-slave-68xk krb5kdc[14082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:19.458956 14100 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:54:19.459944 14100 webserver.cc:460] Webserver started at http://127.30.153.190:36867/ using document root <none> and password file <none>
I20260810 18:54:19.460566 14100 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:19.460636 14100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:19.460873 14100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:19.463135 14100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "3c7a65fe92f1466da3ba33cbe24085e6"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "7798502406dd4804ab92ec096b666fa6"
server_key_iv: "5cbcfe598004402626fbbeb62595124d"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.463764 14100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "3c7a65fe92f1466da3ba33cbe24085e6"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "7798502406dd4804ab92ec096b666fa6"
server_key_iv: "5cbcfe598004402626fbbeb62595124d"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.467742 14100 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:54:19.470391 14121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:19.471872 14100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:54:19.472051 14100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "3c7a65fe92f1466da3ba33cbe24085e6"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "7798502406dd4804ab92ec096b666fa6"
server_key_iv: "5cbcfe598004402626fbbeb62595124d"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.472180 14100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:19.487320 14100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:19.490347 14100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:19.490566 14100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:19.497949 14100 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:38493
I20260810 18:54:19.498274 14204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:38493 every 8 connection(s)
I20260810 18:54:19.498997 14100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:54:19.500594 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 14100
I20260810 18:54:19.500710 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:54:19.501107 31334 external_mini_cluster.cc:1428] Setting key 5db27a0e2cf7622e81b8c623414c458c
I20260810 18:54:19.502241 14206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:19.509047 14206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6: Bootstrap starting.
Aug 10 18:54:19 dist-test-slave-68xk krb5kdc[14082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:54:19.511698 14206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:19.512585 14206 log.cc:824] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:19.514835 14206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6: No bootstrap required, opened a new log
I20260810 18:54:19.516207 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:19.502739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45474 (local address 127.30.153.190:38493)
0810 18:54:19.503180 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:19.503199 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:19.503240 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:54:19.504087 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:19.504110 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:19.504512 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:19.504887 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:19.505588 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:19.507098 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:19.507956 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:19.508643 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:19.508797 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:19.511498 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:19.511503 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:19.511516 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:19.511543 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:19.513430 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:19.513908 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:19.513913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:19.513917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:19.514022 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:19.514307 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:19.514309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:19.514310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:19.514743 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:19.514909 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:19.515195 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 18:54:19.515499 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":125,"threads_started":1}
I20260810 18:54:19.518908 14206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7a65fe92f1466da3ba33cbe24085e6" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38493 } }
I20260810 18:54:19.519122 14206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:19.519201 14206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c7a65fe92f1466da3ba33cbe24085e6, State: Initialized, Role: FOLLOWER
I20260810 18:54:19.519639 14206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7a65fe92f1466da3ba33cbe24085e6" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38493 } }
I20260810 18:54:19.519783 14206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:54:19.519845 14206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:54:19.519963 14206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:19.520823 14206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7a65fe92f1466da3ba33cbe24085e6" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38493 } }
I20260810 18:54:19.521158 14206 leader_election.cc:302] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c7a65fe92f1466da3ba33cbe24085e6; no voters:
I20260810 18:54:19.521452 14206 leader_election.cc:288] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:54:19.521601 14213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:19.521886 14213 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [term 1 LEADER]: Becoming Leader. State: Replica: 3c7a65fe92f1466da3ba33cbe24085e6, State: Running, Role: LEADER
I20260810 18:54:19.522171 14213 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7a65fe92f1466da3ba33cbe24085e6" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38493 } }
I20260810 18:54:19.522729 14206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:54:19.524190 14217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c7a65fe92f1466da3ba33cbe24085e6. Latest consensus state: current_term: 1 leader_uuid: "3c7a65fe92f1466da3ba33cbe24085e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7a65fe92f1466da3ba33cbe24085e6" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38493 } } }
I20260810 18:54:19.524317 14217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:19.524631 14216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c7a65fe92f1466da3ba33cbe24085e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c7a65fe92f1466da3ba33cbe24085e6" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38493 } } }
I20260810 18:54:19.524744 14216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6 [sys.catalog]: This master's current role is: LEADER
I20260810 18:54:19.525365 14236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:54:19.527908 14236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:54:19.533193 14236 catalog_manager.cc:1347] Generated new cluster ID: 935bb789167640f99115e10830c1f868
I20260810 18:54:19.533262 14236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:54:19.543668 14236 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:54:19.544582 14236 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:54:19.550192 14236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6: Generated new TSK 0
I20260810 18:54:19.550984 14236 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:54:19.586287 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38493
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:19.705754 14250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:19.706059 14250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:19.706141 14250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:19.710189 14250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:19.710289 14250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:19.710386 14250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:54:19.715852 14250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:19.716974 14250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:19.718039 14250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:19.728158 14262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:19.728168 14250 server_base.cc:1034] running on GCE node
W20260810 18:54:19.728206 14258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:19.728199 14259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:19.728710 14250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:19.729384 14250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:19.730576 14250 hybrid_clock.cc:648] HybridClock initialized: now 1786388059730559 us; error 27 us; skew 500 ppm
Aug 10 18:54:19 dist-test-slave-68xk krb5kdc[14082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:19.733381 14250 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:54:19.734226 14250 webserver.cc:460] Webserver started at http://127.30.153.129:43165/ using document root <none> and password file <none>
I20260810 18:54:19.734858 14250 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:19.734956 14250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:19.735204 14250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:19.737229 14250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "40d13abecc54429692552eb71b8c416b"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "3a859fd7a9572a03e35fd28c16479176"
server_key_iv: "88bea9e365b960ad5e53086e2fdd1609"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.737799 14250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "40d13abecc54429692552eb71b8c416b"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "3a859fd7a9572a03e35fd28c16479176"
server_key_iv: "88bea9e365b960ad5e53086e2fdd1609"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.741616 14250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:54:19.744251 14270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:19.745478 14250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:54:19.745626 14250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "40d13abecc54429692552eb71b8c416b"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "3a859fd7a9572a03e35fd28c16479176"
server_key_iv: "88bea9e365b960ad5e53086e2fdd1609"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.745740 14250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:19.764357 14250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:19.767581 14250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:19.767795 14250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:19.768390 14250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:19.769228 14250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:19.769301 14250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:19.769373 14250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:19.769428 14250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:19.779050 14250 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:40849
I20260810 18:54:19.779095 14444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:40849 every 8 connection(s)
I20260810 18:54:19.779935 14250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:54:19.784152 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 14250
I20260810 18:54:19.784301 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:54:19.784646 31334 external_mini_cluster.cc:1428] Setting key 10afb5fd837d0029c975f8a63c6dbb5c
Aug 10 18:54:19 dist-test-slave-68xk krb5kdc[14082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:54:19.795043 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:19.781726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:48333 (local address 127.30.153.190:38493)
0810 18:54:19.781899 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:19.781904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:19.782771 (+ 867us) server_negotiation.cc:407] Connection header received
0810 18:54:19.784192 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:19.784196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:19.784256 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:19.784372 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:19.785865 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:19.786544 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:19.787746 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:19.787982 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:19.788094 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:19.790635 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:19.790649 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:19.790655 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:19.790693 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:19.792707 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:19.793222 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:19.793228 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:19.793231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:19.793336 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:19.793692 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:19.793699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:19.793702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:19.793946 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:19.794061 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:19.794672 (+ 611us) server_negotiation.cc:299] Negotiation successful
0810 18:54:19.794827 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:54:19.795842 14447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:19.782087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38493 (local address 127.30.153.129:48333)
0810 18:54:19.782618 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:19.782669 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:19.783976 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:19.784571 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:19.784581 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:19.785109 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:19.785713 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:19.785733 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:19.786690 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:19.786693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:19.787604 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:19.787614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:19.788140 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:19.788144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:19.788314 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:19.788912 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:19.788928 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:19.790453 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:19.792842 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:19.792847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:19.792858 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:19.793114 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:19.793442 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:19.793444 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:19.793445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:19.793576 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:19.794084 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:19.794090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:19.794338 (+ 248us) client_negotiation.cc:769] Sending connection context
0810 18:54:19.794585 (+ 247us) client_negotiation.cc:240] Negotiation successful
0810 18:54:19.794893 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:54:19.797379 14445 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38493
I20260810 18:54:19.797646 14445 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:19.798235 14445 heartbeater.cc:511] Master 127.30.153.190:38493 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
I20260810 18:54:19.800105 14155 ts_manager.cc:194] Registered new tserver with Master: 40d13abecc54429692552eb71b8c416b (127.30.153.129:40849)
I20260810 18:54:19.801318 14155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:48333
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:54:19.818368 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38493
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:19.934620 14450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:19.934865 14450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:19.934950 14450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:19.938923 14450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:19.938977 14450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:19.939033 14450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:54:19.944136 14450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:19.945225 14450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:19.946203 14450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:19.958127 14459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:19.958122 14462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:19.958401 14450 server_base.cc:1034] running on GCE node
W20260810 18:54:19.959178 14460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:19.959764 14450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:19.961530 14450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:19.962790 14450 hybrid_clock.cc:648] HybridClock initialized: now 1786388059962762 us; error 39 us; skew 500 ppm
Aug 10 18:54:19 dist-test-slave-68xk krb5kdc[14082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:19.966219 14450 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:54:19.967159 14450 webserver.cc:460] Webserver started at http://127.30.153.130:34095/ using document root <none> and password file <none>
I20260810 18:54:19.967746 14450 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:19.967809 14450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:19.968026 14450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:19.970103 14450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "23a90479dfef4efcae707cb13a378dee"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "b9d6f5757a98d288ec7aed31c2f0285b"
server_key_iv: "02fef068c757aef33553ce86dc2befb3"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.970700 14450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "23a90479dfef4efcae707cb13a378dee"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "b9d6f5757a98d288ec7aed31c2f0285b"
server_key_iv: "02fef068c757aef33553ce86dc2befb3"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.974560 14450 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:54:19.977202 14471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:19.978608 14450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:54:19.978763 14450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "23a90479dfef4efcae707cb13a378dee"
format_stamp: "Formatted at 2026-08-10 18:54:19 on dist-test-slave-68xk"
server_key: "b9d6f5757a98d288ec7aed31c2f0285b"
server_key_iv: "02fef068c757aef33553ce86dc2befb3"
server_key_version: "encryptionkey@0"
I20260810 18:54:19.978881 14450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:19.995337 14450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:19.998626 14450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:19.998831 14450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:19.999426 14450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:20.000308 14450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:20.000383 14450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:20.000456 14450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:20.000499 14450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:20.010327 14450 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:43145
I20260810 18:54:20.010357 14643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:43145 every 8 connection(s)
I20260810 18:54:20.011399 14450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:54:20.016535 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 14450
I20260810 18:54:20.016707 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:54:20.017052 31334 external_mini_cluster.cc:1428] Setting key 93fcdf5f50b2f8a2c650c71be8da0271
Aug 10 18:54:20 dist-test-slave-68xk krb5kdc[14082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:54:20.027426 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.013717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:60189 (local address 127.30.153.190:38493)
0810 18:54:20.013970 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.013975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.014836 (+ 861us) server_negotiation.cc:407] Connection header received
0810 18:54:20.016365 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.016369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.016445 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.016565 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:20.017948 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.018534 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.019715 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.019943 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.020049 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.023263 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:20.023265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:20.023267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:20.023291 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:20.024978 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:20.025663 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:20.025668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:20.025670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:20.025780 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:20.026187 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:20.026191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:20.026193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:20.026396 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:20.026555 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.027103 (+ 548us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.027242 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:54:20.028314 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.014051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38493 (local address 127.30.153.130:60189)
0810 18:54:20.014659 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:20.014712 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:20.016074 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:20.016772 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:20.016783 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:20.017286 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:20.017791 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.017804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.018660 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.018665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.019574 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.019583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.020106 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.020110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.020288 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.021010 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:20.021038 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:20.023034 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:20.025160 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:20.025169 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:20.025189 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:20.025535 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:20.025888 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:20.025893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:20.025895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:20.026046 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:20.026579 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:20.026586 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:20.026844 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 18:54:20.027054 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 18:54:20.027418 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":134,"threads_started":1}
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:54:20.029613 14644 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38493
I20260810 18:54:20.029898 14644 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:20.030510 14644 heartbeater.cc:511] Master 127.30.153.190:38493 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
I20260810 18:54:20.031818 14155 ts_manager.cc:194] Registered new tserver with Master: 23a90479dfef4efcae707cb13a378dee (127.30.153.130:43145)
I20260810 18:54:20.032428 14155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:60189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:54:20.049968 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38493
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:54:20.163894 14650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:54:20.164188 14650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:54:20.164274 14650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:54:20.168337 14650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:54:20.168421 14650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:54:20.168509 14650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:54:20.173688 14650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:54:20.174810 14650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:54:20.176261 14650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:54:20.186117 14658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:20.186223 14663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:54:20.186127 14659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:54:20.186415 14650 server_base.cc:1034] running on GCE node
I20260810 18:54:20.186796 14650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:54:20.187372 14650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:54:20.188534 14650 hybrid_clock.cc:648] HybridClock initialized: now 1786388060188518 us; error 31 us; skew 500 ppm
Aug 10 18:54:20 dist-test-slave-68xk krb5kdc[14082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:54:20.191514 14650 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:54:20.192663 14650 webserver.cc:460] Webserver started at http://127.30.153.131:46191/ using document root <none> and password file <none>
I20260810 18:54:20.193333 14650 fs_manager.cc:359] Metadata directory not provided
I20260810 18:54:20.193429 14650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:54:20.193650 14650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:54:20.195761 14650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0"
format_stamp: "Formatted at 2026-08-10 18:54:20 on dist-test-slave-68xk"
server_key: "7ec465f7aff966fd2d13da91d3976390"
server_key_iv: "bff2d4bd13c16aab9af847278a825ec1"
server_key_version: "encryptionkey@0"
I20260810 18:54:20.196389 14650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0"
format_stamp: "Formatted at 2026-08-10 18:54:20 on dist-test-slave-68xk"
server_key: "7ec465f7aff966fd2d13da91d3976390"
server_key_iv: "bff2d4bd13c16aab9af847278a825ec1"
server_key_version: "encryptionkey@0"
I20260810 18:54:20.200268 14650 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260810 18:54:20.202888 14678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:20.204129 14650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:54:20.204277 14650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0"
format_stamp: "Formatted at 2026-08-10 18:54:20 on dist-test-slave-68xk"
server_key: "7ec465f7aff966fd2d13da91d3976390"
server_key_iv: "bff2d4bd13c16aab9af847278a825ec1"
server_key_version: "encryptionkey@0"
I20260810 18:54:20.204387 14650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:54:20.225003 14650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:54:20.228524 14650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:54:20.228751 14650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:54:20.229404 14650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:54:20.230258 14650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:54:20.230333 14650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:20.230403 14650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:54:20.230472 14650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:54:20.240222 14650 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:34943
I20260810 18:54:20.240257 14844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:34943 every 8 connection(s)
I20260810 18:54:20.241123 14650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:54:20.247051 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 14650
I20260810 18:54:20.247231 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:54:20.247545 31334 external_mini_cluster.cc:1428] Setting key 54ee4fdd85d34cd70739f0bbf9bd49ba
Aug 10 18:54:20 dist-test-slave-68xk krb5kdc[14082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:54:20.256208 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.243150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:35239 (local address 127.30.153.190:38493)
0810 18:54:20.243315 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.243320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.244259 (+ 939us) server_negotiation.cc:407] Connection header received
0810 18:54:20.245657 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.245660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.245711 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.245815 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:20.247184 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.247918 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.249689 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.249993 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.250090 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.252819 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:20.252822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:20.252824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:20.252847 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:20.254163 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:20.254726 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:20.254728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:20.254730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:20.254809 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:20.255190 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:20.255193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:20.255197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:20.255412 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:20.255494 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.255943 (+ 449us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.256052 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:54:20.256951 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.243600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38493 (local address 127.30.153.131:35239)
0810 18:54:20.244081 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:20.244136 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:20.245448 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:20.245970 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:20.245976 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:20.246316 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:54:20.246980 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.247003 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.248082 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.248086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.249552 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.249567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.250132 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.250134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.250280 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.250975 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:54:20.250991 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:54:20.252667 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:54:20.254270 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:20.254274 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:20.254285 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:20.254635 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:20.254903 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:54:20.254905 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:54:20.254906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:54:20.255034 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:54:20.255513 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:54:20.255518 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:54:20.255728 (+ 210us) client_negotiation.cc:769] Sending connection context
0810 18:54:20.255874 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 18:54:20.256117 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":140,"threads_started":1}
I20260810 18:54:20.258060 14846 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38493
I20260810 18:54:20.258348 14846 heartbeater.cc:464] Registering TS with master...
I20260810 18:54:20.258909 14846 heartbeater.cc:511] Master 127.30.153.190:38493 requested a full tablet report, sending...
I20260810 18:54:20.260010 14155 ts_manager.cc:194] Registered new tserver with Master: b29cfd9b0bbc4c9495f4c7db2fb1f8b0 (127.30.153.131:34943)
I20260810 18:54:20.260764 14155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:35239
I20260810 18:54:20.261620 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:54:20.271741 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.263896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45480 (local address 127.30.153.190:38493)
0810 18:54:20.264051 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.264055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.264245 (+ 190us) server_negotiation.cc:407] Connection header received
0810 18:54:20.264404 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.264409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.264454 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.264539 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:54:20.265292 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.265794 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.266742 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.266964 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.267071 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.268416 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:54:20.268420 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:54:20.268423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:54:20.268452 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:54:20.269908 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:20.270384 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:20.270388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:20.270389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:20.270519 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:54:20.270821 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:54:20.270824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:54:20.270826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:54:20.271053 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:54:20.271208 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.271454 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.271579 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:54:20.275156 14155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:54:20.278519 14155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:54:20.293524 14863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.287648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:40849 (local address 127.0.0.1:51312)
0810 18:54:20.288613 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:20.288633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:20.288773 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:20.289523 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:20.289525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:20.289541 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:54:20.289607 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.289610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.290824 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.290830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.291975 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.291986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.292933 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.292936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.293054 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.293098 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 18:54:20.293218 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 18:54:20.293341 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":90,"threads_started":1}
I20260810 18:54:20.293542 14871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.288092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51312 (local address 127.30.153.129:40849)
0810 18:54:20.288560 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.288576 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.288750 (+ 174us) server_negotiation.cc:407] Connection header received
0810 18:54:20.288915 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.288919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.289068 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.289238 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:54:20.289712 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.290644 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.292106 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.292806 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.292930 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.293063 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.293271 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.293393 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":151,"threads_started":1}
I20260810 18:54:20.294103 14866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.287277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:34943 (local address 127.0.0.1:53570)
0810 18:54:20.288119 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:20.288170 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:20.288305 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:20.289918 (+ 1613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:20.289921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:20.289939 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:54:20.290020 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.290026 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.291359 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.291363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.292518 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.292526 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.293689 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.293692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.293790 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.293818 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:54:20.293898 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 18:54:20.293962 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":24,"thread_start_us":73,"threads_started":1}
I20260810 18:54:20.294349 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.287417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53570 (local address 127.30.153.131:34943)
0810 18:54:20.289456 (+ 2039us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.289464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.289508 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:54:20.289579 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.289583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.289757 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.289915 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:54:20.290141 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.291218 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.292630 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.293370 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.293483 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.293616 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.293952 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.294204 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1956,"thread_start_us":162,"threads_started":1}
I20260810 18:54:20.295087 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.287364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:43145 (local address 127.0.0.1:50746)
0810 18:54:20.288486 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:20.288500 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:20.288580 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:20.289240 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:20.289246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:20.289268 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:54:20.289413 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.289422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.291095 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.291098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.292370 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.292381 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.294667 (+ 2286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.294673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.294768 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.294794 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:54:20.294880 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 18:54:20.294950 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1025,"mutex_wait_us":29,"thread_start_us":104,"threads_started":1}
I20260810 18:54:20.295150 14870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.288097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.30.153.130:43145)
0810 18:54:20.288820 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.288829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.288848 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:54:20.288909 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.288913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.289071 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.289208 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:54:20.289557 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.290803 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.292504 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.293298 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.293416 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.293525 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.294915 (+ 1390us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.295007 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":117,"threads_started":1}
I20260810 18:54:20.296729 14348 tablet_service.cc:1467] Processing CreateTablet for tablet ea85c70e43b1434ea7361fe206617029 (DEFAULT_TABLE table=test-table [id=65a0174e17944452ab7de4f42eea1b5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:54:20.297068 14746 tablet_service.cc:1467] Processing CreateTablet for tablet ea85c70e43b1434ea7361fe206617029 (DEFAULT_TABLE table=test-table [id=65a0174e17944452ab7de4f42eea1b5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:54:20.297930 14348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea85c70e43b1434ea7361fe206617029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:20.298033 14746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea85c70e43b1434ea7361fe206617029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:20.298087 14546 tablet_service.cc:1467] Processing CreateTablet for tablet ea85c70e43b1434ea7361fe206617029 (DEFAULT_TABLE table=test-table [id=65a0174e17944452ab7de4f42eea1b5f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:54:20.299438 14546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ea85c70e43b1434ea7361fe206617029. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:54:20.304188 14872 tablet_bootstrap.cc:492] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Bootstrap starting.
I20260810 18:54:20.305536 14873 tablet_bootstrap.cc:492] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b: Bootstrap starting.
I20260810 18:54:20.306079 14874 tablet_bootstrap.cc:492] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Bootstrap starting.
I20260810 18:54:20.306725 14872 tablet_bootstrap.cc:654] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:20.307579 14873 tablet_bootstrap.cc:654] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:20.308113 14872 log.cc:824] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:20.308411 14873 log.cc:824] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:20.308977 14874 tablet_bootstrap.cc:654] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Neither blocks nor log segments found. Creating new log.
I20260810 18:54:20.309815 14874 log.cc:824] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Log is configured to *not* fsync() on all Append() calls
I20260810 18:54:20.310498 14873 tablet_bootstrap.cc:492] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b: No bootstrap required, opened a new log
I20260810 18:54:20.310710 14873 ts_tablet_manager.cc:1397] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:54:20.310710 14872 tablet_bootstrap.cc:492] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: No bootstrap required, opened a new log
I20260810 18:54:20.310904 14872 ts_tablet_manager.cc:1397] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:54:20.311779 14874 tablet_bootstrap.cc:492] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: No bootstrap required, opened a new log
I20260810 18:54:20.311954 14874 ts_tablet_manager.cc:1397] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:54:20.314019 14873 raft_consensus.cc:348] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.314260 14873 raft_consensus.cc:374] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:20.314324 14873 raft_consensus.cc:729] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40d13abecc54429692552eb71b8c416b, State: Initialized, Role: FOLLOWER
I20260810 18:54:20.314796 14873 consensus_queue.cc:260] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.314785 14872 raft_consensus.cc:348] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.315024 14872 raft_consensus.cc:374] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:20.315091 14872 raft_consensus.cc:729] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b29cfd9b0bbc4c9495f4c7db2fb1f8b0, State: Initialized, Role: FOLLOWER
I20260810 18:54:20.315490 14445 heartbeater.cc:503] Master 127.30.153.190:38493 was elected leader, sending a full tablet report...
I20260810 18:54:20.315537 14872 consensus_queue.cc:260] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.315680 14874 raft_consensus.cc:348] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.315905 14874 raft_consensus.cc:374] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:54:20.315954 14874 raft_consensus.cc:729] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23a90479dfef4efcae707cb13a378dee, State: Initialized, Role: FOLLOWER
I20260810 18:54:20.316381 14874 consensus_queue.cc:260] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.316817 14872 ts_tablet_manager.cc:1428] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:54:20.317130 14873 ts_tablet_manager.cc:1428] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:54:20.317147 14846 heartbeater.cc:503] Master 127.30.153.190:38493 was elected leader, sending a full tablet report...
I20260810 18:54:20.317313 14874 ts_tablet_manager.cc:1428] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 18:54:20.317641 14644 heartbeater.cc:503] Master 127.30.153.190:38493 was elected leader, sending a full tablet report...
I20260810 18:54:20.460361 14879 raft_consensus.cc:482] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:54:20.460577 14879 raft_consensus.cc:504] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.461814 14879 leader_election.cc:288] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b29cfd9b0bbc4c9495f4c7db2fb1f8b0 (127.30.153.131:34943), 23a90479dfef4efcae707cb13a378dee (127.30.153.130:43145)
I20260810 18:54:20.467424 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.462215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:60201 (local address 127.30.153.131:34943)
0810 18:54:20.462327 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.462331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.462711 (+ 380us) server_negotiation.cc:407] Connection header received
0810 18:54:20.462794 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.462796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.462853 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.462957 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:54:20.463370 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.464161 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.466126 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.466759 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.466887 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.466938 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.467225 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.467299 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:54:20.467424 14870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.462188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:43741 (local address 127.30.153.130:43145)
0810 18:54:20.462324 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.462329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.462630 (+ 301us) server_negotiation.cc:407] Connection header received
0810 18:54:20.462801 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.462804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.462856 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.462952 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:54:20.463370 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.464158 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.466126 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.466777 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.466887 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.466939 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.467235 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.467301 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:54:20.467458 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.462087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:43145 (local address 127.30.153.129:43741)
0810 18:54:20.462517 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:20.462553 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:20.462692 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:20.462983 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:20.462988 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:20.463007 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:54:20.463246 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.463253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.464295 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.464298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.465983 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.465995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.466897 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.466901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.467068 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.467141 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 18:54:20.467209 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:54:20.467265 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":252,"spinlock_wait_cycles":6528,"thread_start_us":136,"threads_started":1}
I20260810 18:54:20.467458 14885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.462132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:34943 (local address 127.30.153.129:60201)
0810 18:54:20.462611 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:20.462626 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:54:20.462704 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:54:20.463007 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:54:20.463010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:54:20.463020 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:54:20.463246 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.463253 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.464295 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.464298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.465983 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:54:20.465995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.466897 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.466901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:54:20.467061 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.467092 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:54:20.467180 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:54:20.467265 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":270,"thread_start_us":87,"threads_started":1}
I20260810 18:54:20.468147 14579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea85c70e43b1434ea7361fe206617029" candidate_uuid: "40d13abecc54429692552eb71b8c416b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23a90479dfef4efcae707cb13a378dee" is_pre_election: true
I20260810 18:54:20.468192 14777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea85c70e43b1434ea7361fe206617029" candidate_uuid: "40d13abecc54429692552eb71b8c416b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" is_pre_election: true
I20260810 18:54:20.468451 14579 raft_consensus.cc:2454] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40d13abecc54429692552eb71b8c416b in term 0.
I20260810 18:54:20.468490 14777 raft_consensus.cc:2454] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40d13abecc54429692552eb71b8c416b in term 0.
I20260810 18:54:20.468959 14274 leader_election.cc:302] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23a90479dfef4efcae707cb13a378dee, 40d13abecc54429692552eb71b8c416b; no voters:
I20260810 18:54:20.469254 14879 raft_consensus.cc:2781] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:54:20.469334 14879 raft_consensus.cc:482] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:54:20.469367 14879 raft_consensus.cc:3037] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:20.470481 14879 raft_consensus.cc:504] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.470829 14879 leader_election.cc:288] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [CANDIDATE]: Term 1 election: Requested vote from peers b29cfd9b0bbc4c9495f4c7db2fb1f8b0 (127.30.153.131:34943), 23a90479dfef4efcae707cb13a378dee (127.30.153.130:43145)
I20260810 18:54:20.471185 14777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea85c70e43b1434ea7361fe206617029" candidate_uuid: "40d13abecc54429692552eb71b8c416b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0"
I20260810 18:54:20.471220 14579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ea85c70e43b1434ea7361fe206617029" candidate_uuid: "40d13abecc54429692552eb71b8c416b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23a90479dfef4efcae707cb13a378dee"
I20260810 18:54:20.471290 14777 raft_consensus.cc:3037] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:20.471356 14579 raft_consensus.cc:3037] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:54:20.472386 14579 raft_consensus.cc:2454] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40d13abecc54429692552eb71b8c416b in term 1.
I20260810 18:54:20.472393 14777 raft_consensus.cc:2454] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40d13abecc54429692552eb71b8c416b in term 1.
I20260810 18:54:20.472772 14272 leader_election.cc:302] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 23a90479dfef4efcae707cb13a378dee, 40d13abecc54429692552eb71b8c416b, b29cfd9b0bbc4c9495f4c7db2fb1f8b0; no voters:
I20260810 18:54:20.472968 14879 raft_consensus.cc:2781] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:54:20.473213 14879 raft_consensus.cc:686] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [term 1 LEADER]: Becoming Leader. State: Replica: 40d13abecc54429692552eb71b8c416b, State: Running, Role: LEADER
I20260810 18:54:20.473590 14879 consensus_queue.cc:237] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } }
I20260810 18:54:20.477038 14152 catalog_manager.cc:5495] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b reported cstate change: term changed from 0 to 1, leader changed from <none> to 40d13abecc54429692552eb71b8c416b (127.30.153.129). New cstate: current_term: 1 leader_uuid: "40d13abecc54429692552eb71b8c416b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40d13abecc54429692552eb71b8c416b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40849 } health_report { overall_health: HEALTHY } } }
W20260810 18:54:20.492189 14847 tablet.cc:2369] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:54:20.512645 14645 tablet.cc:2369] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:54:20.529388 14871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.525752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.30.153.129:40849)
0810 18:54:20.525890 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:54:20.525894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:54:20.525907 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:54:20.525986 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:54:20.525988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:54:20.526042 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:54:20.526135 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:54:20.526472 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.526971 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.527877 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:54:20.528089 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:54:20.528185 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:54:20.528524 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:54:20.528938 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:54:20.529039 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 18:54:20.529225 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 18:54:20.529277 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 18:54:20.531251 14446 tablet.cc:2369] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:54:20.535928 14777 raft_consensus.cc:1261] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 1 FOLLOWER]: Refusing update from remote peer 40d13abecc54429692552eb71b8c416b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:54:20.535945 14579 raft_consensus.cc:1261] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 1 FOLLOWER]: Refusing update from remote peer 40d13abecc54429692552eb71b8c416b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:54:20.536718 14886 consensus_queue.cc:1035] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b29cfd9b0bbc4c9495f4c7db2fb1f8b0" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 34943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:54:20.536957 14879 consensus_queue.cc:1035] T ea85c70e43b1434ea7361fe206617029 P 40d13abecc54429692552eb71b8c416b [LEADER]: Connected to new peer: Peer: permanent_uuid: "23a90479dfef4efcae707cb13a378dee" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 43145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:54:20.545150 14895 mvcc.cc:205] Tried to move back new op lower bound from 7317045495947411456 to 7317045495703347200. Current Snapshot: MvccSnapshot[applied={T|T < 7317045495947411456}]
I20260810 18:54:20.554577 14155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:54:20.554773 14155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:54:20.557190 14155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c7a65fe92f1466da3ba33cbe24085e6: Sending DeleteTablet for 3 replicas of tablet ea85c70e43b1434ea7361fe206617029
I20260810 18:54:20.557950 14546 tablet_service.cc:1514] Processing DeleteTablet for tablet ea85c70e43b1434ea7361fe206617029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:54:20 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:50746
I20260810 18:54:20.557992 14746 tablet_service.cc:1514] Processing DeleteTablet for tablet ea85c70e43b1434ea7361fe206617029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:54:20 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:53570
I20260810 18:54:20.558487 14910 tablet_replica.cc:330] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: stopping tablet replica
I20260810 18:54:20.558787 14348 tablet_service.cc:1514] Processing DeleteTablet for tablet ea85c70e43b1434ea7361fe206617029 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:54:20 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:51312
I20260810 18:54:20.558866 14912 tablet_replica.cc:330] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: stopping tablet replica
I20260810 18:54:20.558885 14910 raft_consensus.cc:2229] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:54:20.559180 14912 raft_consensus.cc:2229] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:54:20.559454 14910 raft_consensus.cc:2258] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:54:20.559484 14912 raft_consensus.cc:2258] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:54:20.559882 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 14250
I20260810 18:54:20.561887 14912 ts_tablet_manager.cc:1905] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:54:20.565047 14912 ts_tablet_manager.cc:1918] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:54:20.565171 14912 log.cc:1196] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/ea85c70e43b1434ea7361fe206617029
I20260810 18:54:20.565555 14912 ts_tablet_manager.cc:1939] T ea85c70e43b1434ea7361fe206617029 P b29cfd9b0bbc4c9495f4c7db2fb1f8b0: Deleting consensus metadata
I20260810 18:54:20.566844 14127 catalog_manager.cc:4841] TS b29cfd9b0bbc4c9495f4c7db2fb1f8b0 (127.30.153.131:34943): tablet ea85c70e43b1434ea7361fe206617029 (table test-table [id=65a0174e17944452ab7de4f42eea1b5f]) successfully deleted
I20260810 18:54:20.567230 14910 ts_tablet_manager.cc:1905] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:54:20.569402 14910 ts_tablet_manager.cc:1918] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:54:20.569522 14910 log.cc:1196] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/ea85c70e43b1434ea7361fe206617029
I20260810 18:54:20.569846 14910 ts_tablet_manager.cc:1939] T ea85c70e43b1434ea7361fe206617029 P 23a90479dfef4efcae707cb13a378dee: Deleting consensus metadata
I20260810 18:54:20.570991 14133 catalog_manager.cc:4841] TS 23a90479dfef4efcae707cb13a378dee (127.30.153.130:43145): tablet ea85c70e43b1434ea7361fe206617029 (table test-table [id=65a0174e17944452ab7de4f42eea1b5f]) successfully deleted
W20260810 18:54:20.571689 14126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:40849 (error 108)
I20260810 18:54:20.571791 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 14450
I20260810 18:54:20.581352 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:54:20.580839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:40849 (local address 127.0.0.1:51338)
0810 18:54:20.581072 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 18:54:20.581204 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:40849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 18:54:20.581768 14126 catalog_manager.cc:4584] TS 40d13abecc54429692552eb71b8c416b (127.30.153.129:40849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ea85c70e43b1434ea7361fe206617029: Network error: Client connection negotiation failed: client connection to 127.30.153.129:40849: connect: Connection refused (error 111)
I20260810 18:54:20.590085 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 14650
I20260810 18:54:20.601472 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 14100
2026-08-10T18:54:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2691 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:54:20 dist-test-slave-68xk krb5kdc[14917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:54:20 dist-test-slave-68xk krb5kdc[14917](info): set up 2 sockets
Aug 10 18:54:20 dist-test-slave-68xk krb5kdc[14917](info): commencing operation
krb5kdc: starting...
W20260810 18:54:21.879058 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:54:21 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:54:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:54:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.153.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.153.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/rangeradmin_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/rangeradmin_127.30.153.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.153.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.153.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/rangerlookup_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/rangerlookup_127.30.153.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.153.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.153.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/HTTP_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/HTTP_127.30.153.148@KRBTEST.COM.keytab.
I20260810 18:54:22.073544 31334 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBp6AcV/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/postgres -l logfile start
2026-08-10 18:54:24.733 UTC [14965] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 18:54:24.734 UTC [14965] LOG: listening on IPv4 address "127.30.153.148", port 35785
2026-08-10 18:54:24.737 UTC [14965] LOG: listening on Unix socket "/tmp/.s.PGSQL.35785"
2026-08-10 18:54:24.759 UTC [14972] LOG: database system was shut down at 2026-08-10 18:54:23 UTC
2026-08-10 18:54:24.763 UTC [14965] LOG: database system is ready to accept connections
I20260810 18:54:26.025933 31334 mini_postgres.cc:96] Postgres bound to 35785
2026-08-10 18:54:26.033 UTC [14988] FATAL: database "slave" does not exist
127.30.153.148:35785 - accepting connections
I20260810 18:54:26.034550 31334 mini_ranger.cc:161] Starting Ranger...
I20260810 18:54:26.051182 31334 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 18:54:26.582228 31334 mini_ranger.cc:87] Created ranger Postgres database
I20260810 18:54:26.582340 31334 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-admin
2026-08-10 18:54:26,864 [I] DB FLAVOR :POSTGRES
2026-08-10 18:54:26,865 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:54:26,865 [I] Checking connection..
2026-08-10 18:54:26,865 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 18:54:27,147 [I] Checking connection passed.
2026-08-10 18:54:27,147 [I] --------- Verifying version history table ---------
2026-08-10 18:54:27,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:54:27,400 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:54:27,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:54:27,663 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:54:27,664 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 18:54:27,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 18:54:27.922 UTC [15090] WARNING: there is no transaction in progress
2026-08-10 18:54:27,931 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 18:54:27,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:54:28,181 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 18:54:28,181 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:54:28,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 18:54:28,450 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 18:54:28,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-68xk.c.gcp-upstream.internal','N') ;"
2026-08-10 18:54:28,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:54:29,194 [I] Table x_portal_user does not exist in database ranger
2026-08-10 18:54:29,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:54:29,457 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 18:54:29,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:54:29,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:54:29,980 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 18:54:29,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 18:54:31.038 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.057 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.076 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.093 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.111 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.134 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.193 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.200 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.217 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.228 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.238 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.253 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.263 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.271 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.278 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:31.286 UTC [15263] WARNING: there is no transaction in progress
2026-08-10 18:54:32,041 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 18:54:32,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 18:54:32,345 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 18:54:32,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:54:32,594 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 18:54:32,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:54:32,857 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 18:54:32,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:54:33,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:54:33,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-68xk.c.gcp-upstream.internal';"
2026-08-10 18:54:33,668 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 18:54:33,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 18:54:33,917 [I] DB_PATCHES have already been applied
I20260810 18:54:33.923540 31334 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBp6AcV/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 18:54:33.923614 31334 mini_ranger.cc:193] Using host: 127.30.153.148
I20260810 18:54:33.928576 31334 mini_ranger.cc:227] Ranger admin URL: http://127.30.153.148:49267
Aug 10, 2026 6:54:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:54:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:54:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:54:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:54:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:54:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:54:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 18:54:35.342585 31334 mini_ranger.cc:160] Time spent starting Ranger: real 9.308s user 0.009s sys 0.005s
Aug 10, 2026 6:54:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:54:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 10, 2026 6:54:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:54:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:54:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 18:54:58.039559 31334 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.153.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.153.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/rangerkms_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/rangerkms_127.30.153.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.153.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.153.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.153.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/HTTP_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.153.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/HTTP_127.30.153.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 18:54:58 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388098, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 18:54:58.141223 31334 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 18:54:58.152562 31334 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 18:54:59.117151 31334 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 18:54:59.117250 31334 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms
I20260810 18:54:59.117358 31334 mini_ranger_kms.cc:227] Using postgres at 127.30.153.148:35785
2026-08-10 18:54:59,372 [I] DB FLAVOR :POSTGRES
2026-08-10 18:54:59,373 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:54:59,373 [I] Checking connection
2026-08-10 18:54:59,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 18:54:59,645 [I] connection success
2026-08-10 18:54:59,645 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 18:54:59,645 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 18:54:59,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 18:54:59,900 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 18:54:59,900 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:54:59,901 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 18:54:59,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:35785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 18:55:00,180 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 18:55:00.679211 31334 mini_ranger_kms.cc:310] Created kms service
I20260810 18:55:00.904313 31334 mini_ranger_kms.cc:326] Created kudu user
I20260810 18:55:00.952203 31334 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 18:55:00 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388098, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:01.376188 31334 mini_ranger_kms.cc:384] Added ranger policy
I20260810 18:55:01.376358 31334 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 18:55:01.376382 31334 mini_ranger_kms.cc:240] Using host: 127.30.153.148
I20260810 18:55:01.381289 31334 mini_ranger_kms.cc:276] Ranger KMS PID: 15546
I20260810 18:55:01.381438 31334 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.153.148:48137
Aug 10, 2026 6:55:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:55:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:55:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:55:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:55:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:55:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 18:55:03 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786388103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.153.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:04.179934 31334 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.039s user 0.010s sys 0.226s
I20260810 18:55:04.180085 31334 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 18:55:04.180208 31334 mini_ranger_kms.cc:401] 127.30.153.148:48137/kms/v1/keys
Aug 10 18:55:04 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.153.148@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
Aug 10 18:55:04 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.153.148@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
Aug 10 18:55:07 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:55:07.055104 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42097
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:42097
--ranger_config_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:48137/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:55:07.191916 15627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:55:07.192225 15627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:55:07.192271 15627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:55:07.196902 15627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:55:07.196964 15627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:55:07.196985 15627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:55:07.197000 15627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:55:07.197039 15627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:55:07.202745 15627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.153.148:48137/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:42097
--ranger_config_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:55:07.203874 15627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:55:07.204963 15627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:55:07.214479 15627 server_base.cc:1034] running on GCE node
W20260810 18:55:07.214387 15643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:07.214457 15638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:07.214392 15639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:07.215319 15627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:55:07.216557 15627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:55:07.217772 15627 hybrid_clock.cc:648] HybridClock initialized: now 1786388107217742 us; error 42 us; skew 500 ppm
Aug 10 18:55:07 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388107, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:07.225988 15627 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:55:07.227169 15627 webserver.cc:460] Webserver started at http://127.30.153.190:35877/ using document root <none> and password file <none>
I20260810 18:55:07.227818 15627 fs_manager.cc:359] Metadata directory not provided
I20260810 18:55:07.227913 15627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:55:07.228171 15627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:55:07 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388107, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:07.291368 15627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "6f45e215049c485bb5cce1b79519b02c"
format_stamp: "Formatted at 2026-08-10 18:55:07 on dist-test-slave-68xk"
server_key: "ea1179aa46228b0af585c38a918684e2"
server_key_iv: "9639c304424c0a5d779d783e864b254f"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:07.292054 15627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "6f45e215049c485bb5cce1b79519b02c"
format_stamp: "Formatted at 2026-08-10 18:55:07 on dist-test-slave-68xk"
server_key: "ea1179aa46228b0af585c38a918684e2"
server_key_iv: "9639c304424c0a5d779d783e864b254f"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:07.296571 15627 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:55:07.315141 15653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:07.316654 15627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:55:07.316810 15627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "6f45e215049c485bb5cce1b79519b02c"
format_stamp: "Formatted at 2026-08-10 18:55:07 on dist-test-slave-68xk"
server_key: "ea1179aa46228b0af585c38a918684e2"
server_key_iv: "9639c304424c0a5d779d783e864b254f"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:07.316952 15627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:55:07.330981 15627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:55:07.334172 15627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:55:07.334393 15627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:55:07.342669 15627 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:42097
I20260810 18:55:07.342689 15730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:42097 every 8 connection(s)
I20260810 18:55:07.343691 15627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:55:07.345681 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 15627
I20260810 18:55:07.345844 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:55:07.346166 31334 external_mini_cluster.cc:1428] Setting key c03b53806c08a120dfafe9a0bbacaec8
I20260810 18:55:07.347597 15731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:55:07.356161 15731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c: Bootstrap starting.
Aug 10 18:55:07 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:55:07.359501 15731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c: Neither blocks nor log segments found. Creating new log.
I20260810 18:55:07.361074 15731 log.cc:824] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:55:07.363444 15731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c: No bootstrap required, opened a new log
I20260810 18:55:07.363958 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:07.347769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.30.153.190:42097)
0810 18:55:07.348210 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:07.348229 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:07.350810 (+ 2581us) server_negotiation.cc:407] Connection header received
0810 18:55:07.351649 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:07.351667 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:07.352016 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:07.352439 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:07.353251 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:07.354277 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:07.355206 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:07.355745 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:07.355926 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:07.358695 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:07.358701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:07.358715 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:07.358744 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:07.360694 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:07.361189 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:07.361197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:07.361203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:07.361335 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:07.361700 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:07.361703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:07.361704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:07.362113 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:07.362284 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:07.362713 (+ 429us) server_negotiation.cc:299] Negotiation successful
0810 18:55:07.363068 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":156,"threads_started":1}
I20260810 18:55:07.366757 15731 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45e215049c485bb5cce1b79519b02c" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42097 } }
I20260810 18:55:07.366977 15731 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:55:07.367039 15731 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f45e215049c485bb5cce1b79519b02c, State: Initialized, Role: FOLLOWER
I20260810 18:55:07.367494 15731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45e215049c485bb5cce1b79519b02c" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42097 } }
I20260810 18:55:07.367620 15731 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:55:07.367688 15731 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:55:07.367795 15731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:55:07.368809 15731 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45e215049c485bb5cce1b79519b02c" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42097 } }
I20260810 18:55:07.369105 15731 leader_election.cc:302] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f45e215049c485bb5cce1b79519b02c; no voters:
I20260810 18:55:07.369382 15731 leader_election.cc:288] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:55:07.369535 15737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:55:07.369845 15737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [term 1 LEADER]: Becoming Leader. State: Replica: 6f45e215049c485bb5cce1b79519b02c, State: Running, Role: LEADER
I20260810 18:55:07.370158 15737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45e215049c485bb5cce1b79519b02c" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42097 } }
I20260810 18:55:07.370712 15731 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:55:07.371968 15740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f45e215049c485bb5cce1b79519b02c. Latest consensus state: current_term: 1 leader_uuid: "6f45e215049c485bb5cce1b79519b02c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45e215049c485bb5cce1b79519b02c" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42097 } } }
I20260810 18:55:07.372088 15740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [sys.catalog]: This master's current role is: LEADER
I20260810 18:55:07.372052 15739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f45e215049c485bb5cce1b79519b02c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f45e215049c485bb5cce1b79519b02c" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42097 } } }
I20260810 18:55:07.372161 15739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c [sys.catalog]: This master's current role is: LEADER
I20260810 18:55:07.372743 15749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:55:07.373340 15731 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:55:07.378914 15749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:55:07.386350 15749 catalog_manager.cc:1347] Generated new cluster ID: fd8de05f00ce430e859238dc903a6c6a
I20260810 18:55:07.386444 15749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:55:07.395215 15749 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:55:07.396493 15749 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:55:07.417521 15749 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c: Generated new TSK 0
I20260810 18:55:07.418469 15749 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:55:08.071589 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:48137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42097
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:55:08.208415 15793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:55:08.208727 15793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:55:08.208823 15793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:55:08.213233 15793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:55:08.213325 15793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:55:08.213415 15793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:55:08.219921 15793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.153.148:48137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:55:08.221544 15793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:55:08.222941 15793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:55:08.235565 15807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:08.235652 15803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:08.235930 15805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:08.236565 15793 server_base.cc:1034] running on GCE node
I20260810 18:55:08.237025 15793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:55:08.237797 15793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:55:08.239001 15793 hybrid_clock.cc:648] HybridClock initialized: now 1786388108238970 us; error 42 us; skew 500 ppm
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:08.242218 15793 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:55:08.243324 15793 webserver.cc:460] Webserver started at http://127.30.153.129:44327/ using document root <none> and password file <none>
I20260810 18:55:08.243948 15793 fs_manager.cc:359] Metadata directory not provided
I20260810 18:55:08.244040 15793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:55:08.244266 15793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:08.265023 15793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "5766da1f194841829d266479b6e94f33"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "50d140d04a1192fe074e79d56dd57979"
server_key_iv: "0041de8e991d80965dd76a3bddb8ff95"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:08.265853 15793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "5766da1f194841829d266479b6e94f33"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "50d140d04a1192fe074e79d56dd57979"
server_key_iv: "0041de8e991d80965dd76a3bddb8ff95"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:08.272325 15793 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260810 18:55:08.302932 15815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:08.304854 15793 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.003s sys 0.001s
I20260810 18:55:08.305013 15793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "5766da1f194841829d266479b6e94f33"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "50d140d04a1192fe074e79d56dd57979"
server_key_iv: "0041de8e991d80965dd76a3bddb8ff95"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:08.305192 15793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:55:08.343901 15793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:55:08.347607 15793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:55:08.347798 15793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:55:08.348518 15793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:55:08.349650 15793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:55:08.349716 15793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:08.349768 15793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:55:08.349793 15793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:08.362227 15793 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:44199
I20260810 18:55:08.362581 15990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:44199 every 8 connection(s)
I20260810 18:55:08.363480 15793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:55:08.372543 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 15793
I20260810 18:55:08.372717 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:55:08.373090 31334 external_mini_cluster.cc:1428] Setting key 7afb6afa603bb8d42d6453ff47ff5353
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:55:08.381033 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:08.365467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:42113 (local address 127.30.153.190:42097)
0810 18:55:08.365794 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:08.365804 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:08.366586 (+ 782us) server_negotiation.cc:407] Connection header received
0810 18:55:08.367580 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:08.367583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:08.367654 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:08.367767 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:08.368855 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:08.369408 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:08.370720 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:08.371052 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:08.371167 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:08.374013 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:08.374017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:08.374019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:08.374047 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:08.378060 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:08.378835 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:08.378842 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:08.378846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:08.378950 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:08.379412 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:08.379419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:08.379424 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:08.379686 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:08.379778 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:08.380566 (+ 788us) server_negotiation.cc:299] Negotiation successful
0810 18:55:08.380749 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":127,"threads_started":1}
I20260810 18:55:08.383069 15996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:08.365929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42097 (local address 127.30.153.129:42113)
0810 18:55:08.366402 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:08.366489 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:08.367370 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:08.367944 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:08.367951 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:08.368232 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:55:08.368706 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:08.368719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:08.369539 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:08.369545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:08.370583 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:08.370593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:08.371207 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:08.371213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:08.371393 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:08.372128 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:55:08.372145 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:55:08.373837 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:55:08.378220 (+ 4383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:08.378229 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:08.378249 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:08.378720 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:08.379066 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:08.379072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:08.379076 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:08.379292 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:08.379856 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:55:08.379865 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:55:08.380248 (+ 383us) client_negotiation.cc:769] Sending connection context
0810 18:55:08.381264 (+ 1016us) client_negotiation.cc:240] Negotiation successful
0810 18:55:08.381688 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":145,"threads_started":1}
I20260810 18:55:08.384939 15991 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42097
I20260810 18:55:08.385267 15991 heartbeater.cc:464] Registering TS with master...
I20260810 18:55:08.385958 15991 heartbeater.cc:511] Master 127.30.153.190:42097 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:55:08.387997 15679 ts_manager.cc:194] Registered new tserver with Master: 5766da1f194841829d266479b6e94f33 (127.30.153.129:44199)
I20260810 18:55:08.389648 15679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:42113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:55:08.409701 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:48137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42097
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 18:55:08.543561 16002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:55:08.543859 16002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:55:08.543964 16002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:55:08.548563 16002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:55:08.548656 16002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:55:08.548784 16002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:55:08.554634 16002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.153.148:48137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:55:08.555848 16002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:55:08.556936 16002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:55:08.569341 16015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:08.569231 16018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:08.569232 16016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:08.570647 16002 server_base.cc:1034] running on GCE node
I20260810 18:55:08.571139 16002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:55:08.571934 16002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:55:08.573132 16002 hybrid_clock.cc:648] HybridClock initialized: now 1786388108573067 us; error 90 us; skew 500 ppm
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:08.576109 16002 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:55:08.577215 16002 webserver.cc:460] Webserver started at http://127.30.153.130:43755/ using document root <none> and password file <none>
I20260810 18:55:08.577840 16002 fs_manager.cc:359] Metadata directory not provided
I20260810 18:55:08.577939 16002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:55:08.578162 16002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:08.593566 16002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "5f5762c8d61b4e519d05df29e47a769b"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "96431c255adfecedcb094f216e223c25"
server_key_iv: "b6f576e1cb1ff67a43a491d004b73d42"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:08.594692 16002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "5f5762c8d61b4e519d05df29e47a769b"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "96431c255adfecedcb094f216e223c25"
server_key_iv: "b6f576e1cb1ff67a43a491d004b73d42"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:08.602322 16002 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.001s
I20260810 18:55:08.622972 16030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:08.624861 16002 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260810 18:55:08.626920 16002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "5f5762c8d61b4e519d05df29e47a769b"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "96431c255adfecedcb094f216e223c25"
server_key_iv: "b6f576e1cb1ff67a43a491d004b73d42"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:08.627148 16002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:55:08.665863 16002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:55:08.674294 16002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:55:08.674556 16002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:55:08.675305 16002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:55:08.676429 16002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:55:08.676503 16002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:08.676577 16002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:55:08.676611 16002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:08.700270 16002 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:33387
I20260810 18:55:08.700438 16197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:33387 every 8 connection(s)
I20260810 18:55:08.701431 16002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:55:08.702669 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 16002
I20260810 18:55:08.702808 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:55:08.703119 31334 external_mini_cluster.cc:1428] Setting key bc69360f70f5c6c7e123650b4408160f
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:55:08.725826 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:08.705890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:56755 (local address 127.30.153.190:42097)
0810 18:55:08.706014 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:08.706019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:08.711504 (+ 5485us) server_negotiation.cc:407] Connection header received
0810 18:55:08.712824 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:08.712829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:08.712886 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:08.712942 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:08.714823 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:08.715684 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:08.717440 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:08.717811 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:08.717891 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:08.721214 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:08.721218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:08.721221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:08.721253 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:08.723178 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:08.723902 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:08.723906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:08.723908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:08.724000 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:08.724423 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:08.724427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:08.724429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:08.724666 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:08.724753 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:08.725499 (+ 746us) server_negotiation.cc:299] Negotiation successful
0810 18:55:08.725644 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:55:08.727633 16204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:08.704314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42097 (local address 127.30.153.130:56755)
0810 18:55:08.711311 (+ 6997us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:08.711383 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:08.712549 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:08.713404 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:08.713414 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:08.713798 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:55:08.714565 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:08.714587 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:08.715855 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:08.715858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:08.717294 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:08.717306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:08.718110 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:08.718114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:08.718339 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:08.719063 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:55:08.719086 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:55:08.720969 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:55:08.723303 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:08.723310 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:08.723327 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:08.723782 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:08.724106 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:08.724109 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:08.724111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:08.724307 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:08.724829 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:55:08.724835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:55:08.725205 (+ 370us) client_negotiation.cc:769] Sending connection context
0810 18:55:08.726083 (+ 878us) client_negotiation.cc:240] Negotiation successful
0810 18:55:08.726498 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6719,"thread_start_us":155,"threads_started":1}
I20260810 18:55:08.729100 16199 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42097
I20260810 18:55:08.729436 16199 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
I20260810 18:55:08.730206 16199 heartbeater.cc:511] Master 127.30.153.190:42097 requested a full tablet report, sending...
I20260810 18:55:08.731663 15679 ts_manager.cc:194] Registered new tserver with Master: 5f5762c8d61b4e519d05df29e47a769b (127.30.153.130:33387)
I20260810 18:55:08.732597 15679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:56755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:55:08.766919 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:48137/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42097
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:46613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
W20260810 18:55:08.949295 16207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:55:08.949627 16207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:55:08.949721 16207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:55:08.953965 16207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:55:08.954066 16207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:55:08.954178 16207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:55:08.959777 16207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.153.148:48137/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42097
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:55:08.960969 16207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:55:08.962086 16207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:55:08.972978 16215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:08.973305 16214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:08.973413 16207 server_base.cc:1034] running on GCE node
W20260810 18:55:08.972903 16218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:08.973991 16207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:55:08.974776 16207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:55:08.975961 16207 hybrid_clock.cc:648] HybridClock initialized: now 1786388108975930 us; error 44 us; skew 500 ppm
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:08.978940 16207 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:55:08.979934 16207 webserver.cc:460] Webserver started at http://127.30.153.131:37785/ using document root <none> and password file <none>
I20260810 18:55:08.980710 16207 fs_manager.cc:359] Metadata directory not provided
I20260810 18:55:08.980823 16207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:55:08.981113 16207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:55:08 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:09.003846 16207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "0ce5c17a57e24ab2831a24381d328b19"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "3d15981d240a8ad06a58b6e778804eb0"
server_key_iv: "6038a1365a67f6bcb41cc526e0c958e0"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:09.004743 16207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "0ce5c17a57e24ab2831a24381d328b19"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "3d15981d240a8ad06a58b6e778804eb0"
server_key_iv: "6038a1365a67f6bcb41cc526e0c958e0"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:09.011538 16207 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260810 18:55:09.030190 16227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:09.031817 16207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:55:09.032053 16207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "0ce5c17a57e24ab2831a24381d328b19"
format_stamp: "Formatted at 2026-08-10 18:55:08 on dist-test-slave-68xk"
server_key: "3d15981d240a8ad06a58b6e778804eb0"
server_key_iv: "6038a1365a67f6bcb41cc526e0c958e0"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:09.032223 16207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:55:09.068776 16207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:55:09.072400 16207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:55:09.072669 16207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:55:09.073357 16207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:55:09.074339 16207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:55:09.074416 16207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:09.074513 16207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:55:09.074548 16207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:09.086552 16395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:37727 every 8 connection(s)
I20260810 18:55:09.086889 16207 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:37727
I20260810 18:55:09.087872 16207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:55:09.096525 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 16207
I20260810 18:55:09.096690 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:55:09.097082 31334 external_mini_cluster.cc:1428] Setting key 173fb2370e20a0fa40729ccd52aa649a
Aug 10 18:55:09 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:55:09.104197 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.090077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:46171 (local address 127.30.153.190:42097)
0810 18:55:09.090227 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:09.090233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:09.091288 (+ 1055us) server_negotiation.cc:407] Connection header received
0810 18:55:09.092286 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:09.092289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:09.092347 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:09.092459 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:09.093584 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.094118 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.095490 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.095728 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.095844 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.100442 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:09.100445 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:09.100447 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:09.100472 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:09.101987 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:09.102585 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:09.102588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:09.102589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:09.102670 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:09.103039 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:09.103042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:09.103044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:09.103248 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:09.103364 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:09.103894 (+ 530us) server_negotiation.cc:299] Negotiation successful
0810 18:55:09.104023 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:55:09.105194 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.090601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42097 (local address 127.30.153.131:46171)
0810 18:55:09.091127 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.091181 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:09.092062 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:09.092656 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:09.092664 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:09.092949 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:55:09.093431 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.093444 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.094248 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.094251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.095357 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.095367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.095890 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.095894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.096075 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.096869 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:55:09.096892 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:55:09.100266 (+ 3374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:55:09.102121 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:09.102127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:09.102143 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:09.102469 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:09.102766 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:09.102770 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:09.102772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:09.102924 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:09.103374 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:55:09.103383 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:55:09.103641 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 18:55:09.103831 (+ 190us) client_negotiation.cc:240] Negotiation successful
0810 18:55:09.104168 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":159,"threads_started":1}
I20260810 18:55:09.106350 16399 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42097
I20260810 18:55:09.106657 16399 heartbeater.cc:464] Registering TS with master...
I20260810 18:55:09.107177 16399 heartbeater.cc:511] Master 127.30.153.190:42097 requested a full tablet report, sending...
I20260810 18:55:09.108240 15679 ts_manager.cc:194] Registered new tserver with Master: 0ce5c17a57e24ab2831a24381d328b19 (127.30.153.131:37727)
I20260810 18:55:09.108816 15679 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:46171
I20260810 18:55:09.112314 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:55:09.123797 15995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.114752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44068 (local address 127.30.153.190:42097)
0810 18:55:09.114905 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:09.114909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:09.115204 (+ 295us) server_negotiation.cc:407] Connection header received
0810 18:55:09.115347 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:09.115350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:09.115403 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:09.115488 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:09.116230 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.116769 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.117646 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.117887 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.118003 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.119974 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:09.119979 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:09.119981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:09.120014 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:09.121856 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:09.122312 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:09.122317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:09.122319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:09.122471 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:09.122772 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:09.122775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:09.122777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:09.123027 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:09.123206 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:09.123468 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 18:55:09.123606 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:55:09.129528 15679 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:55:09.133654 15679 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:55:09.152920 16415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.143729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:37727 (local address 127.0.0.1:56760)
0810 18:55:09.146800 (+ 3071us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.146818 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:09.146904 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:09.147567 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:09.147570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:09.147586 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:55:09.147704 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.147711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.148995 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.148999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.150940 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.150947 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.152267 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.152270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.152414 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.152463 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 18:55:09.152580 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 18:55:09.152724 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3000,"mutex_wait_us":435,"thread_start_us":74,"threads_started":1}
I20260810 18:55:09.153545 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.145105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.30.153.131:37727)
0810 18:55:09.146647 (+ 1542us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:09.146658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:09.146995 (+ 337us) server_negotiation.cc:407] Connection header received
0810 18:55:09.147097 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:09.147103 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:09.147282 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:09.147422 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:55:09.147838 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.148866 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.151078 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.151922 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.152046 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.152202 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:09.153302 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0810 18:55:09.153413 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1439,"thread_start_us":139,"threads_started":1}
I20260810 18:55:09.153905 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.144695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46728 (local address 127.30.153.129:44199)
0810 18:55:09.146350 (+ 1655us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:09.146354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:09.146374 (+ 20us) server_negotiation.cc:407] Connection header received
0810 18:55:09.146498 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:09.146503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:09.146736 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:09.146898 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:55:09.147409 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.149386 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.150948 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.152783 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.152898 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.153038 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:09.153646 (+ 608us) server_negotiation.cc:299] Negotiation successful
0810 18:55:09.153754 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1563,"thread_start_us":111,"threads_started":1}
I20260810 18:55:09.158567 16271 tablet_service.cc:1467] Processing CreateTablet for tablet 7390d6f13222420a8134c8c4101ebf50 (DEFAULT_TABLE table=test-table [id=2d27d7205a7a428aa357fa7631c77718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:55:09.159934 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.144844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:33387 (local address 127.0.0.1:42606)
0810 18:55:09.150123 (+ 5279us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.150138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:09.150588 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:09.154654 (+ 4066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:09.154656 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:09.154671 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:55:09.154737 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.154740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.156982 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.156985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.158161 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.158173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.159505 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.159509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.159616 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.159643 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:55:09.159719 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 18:55:09.159792 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5205,"thread_start_us":76,"threads_started":1}
I20260810 18:55:09.159965 16271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7390d6f13222420a8134c8c4101ebf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:55:09.161345 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.145267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.30.153.130:33387)
0810 18:55:09.146225 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:09.146244 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:09.150233 (+ 3989us) server_negotiation.cc:407] Connection header received
0810 18:55:09.153891 (+ 3658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:09.153900 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:09.154294 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:09.154462 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:55:09.154857 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.156842 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.158335 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.159185 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.159310 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.159438 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:09.160196 (+ 758us) server_negotiation.cc:299] Negotiation successful
0810 18:55:09.161195 (+ 999us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840,"thread_start_us":74,"threads_started":1}
I20260810 18:55:09.164644 16101 tablet_service.cc:1467] Processing CreateTablet for tablet 7390d6f13222420a8134c8c4101ebf50 (DEFAULT_TABLE table=test-table [id=2d27d7205a7a428aa357fa7631c77718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:55:09.153530 16419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.143637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:44199 (local address 127.0.0.1:46728)
0810 18:55:09.145743 (+ 2106us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.145798 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:09.145921 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:09.147086 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:09.147104 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:09.147128 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:55:09.147262 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.147269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.149520 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.149523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.150823 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.150831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.153122 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.153126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.153220 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.153250 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 18:55:09.153325 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 18:55:09.153391 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1896,"mutex_wait_us":26,"thread_start_us":126,"threads_started":1}
I20260810 18:55:09.165892 16101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7390d6f13222420a8134c8c4101ebf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:55:09.168634 15898 tablet_service.cc:1467] Processing CreateTablet for tablet 7390d6f13222420a8134c8c4101ebf50 (DEFAULT_TABLE table=test-table [id=2d27d7205a7a428aa357fa7631c77718]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:55:09.170011 15898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7390d6f13222420a8134c8c4101ebf50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:55:09.172910 16424 tablet_bootstrap.cc:492] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: Bootstrap starting.
I20260810 18:55:09.176028 16424 tablet_bootstrap.cc:654] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: Neither blocks nor log segments found. Creating new log.
I20260810 18:55:09.176995 16425 tablet_bootstrap.cc:492] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: Bootstrap starting.
I20260810 18:55:09.177032 16424 log.cc:824] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: Log is configured to *not* fsync() on all Append() calls
I20260810 18:55:09.179569 16424 tablet_bootstrap.cc:492] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: No bootstrap required, opened a new log
I20260810 18:55:09.179821 16424 ts_tablet_manager.cc:1397] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:55:09.179981 16425 tablet_bootstrap.cc:654] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: Neither blocks nor log segments found. Creating new log.
I20260810 18:55:09.180845 16425 log.cc:824] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:55:09.184033 16425 tablet_bootstrap.cc:492] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: No bootstrap required, opened a new log
I20260810 18:55:09.184226 16425 ts_tablet_manager.cc:1397] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:55:09.184013 16424 raft_consensus.cc:348] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.184294 16424 raft_consensus.cc:374] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:55:09.184350 16424 raft_consensus.cc:729] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ce5c17a57e24ab2831a24381d328b19, State: Initialized, Role: FOLLOWER
I20260810 18:55:09.185107 16424 consensus_queue.cc:260] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.186297 16424 ts_tablet_manager.cc:1428] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:55:09.186751 16399 heartbeater.cc:503] Master 127.30.153.190:42097 was elected leader, sending a full tablet report...
I20260810 18:55:09.186815 16427 tablet_bootstrap.cc:492] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33: Bootstrap starting.
I20260810 18:55:09.188117 16425 raft_consensus.cc:348] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.188372 16425 raft_consensus.cc:374] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:55:09.188427 16425 raft_consensus.cc:729] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f5762c8d61b4e519d05df29e47a769b, State: Initialized, Role: FOLLOWER
I20260810 18:55:09.188911 16425 consensus_queue.cc:260] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.189292 16427 tablet_bootstrap.cc:654] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33: Neither blocks nor log segments found. Creating new log.
I20260810 18:55:09.190114 16425 ts_tablet_manager.cc:1428] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:55:09.190167 16427 log.cc:824] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33: Log is configured to *not* fsync() on all Append() calls
I20260810 18:55:09.190486 16199 heartbeater.cc:503] Master 127.30.153.190:42097 was elected leader, sending a full tablet report...
I20260810 18:55:09.201012 16427 tablet_bootstrap.cc:492] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33: No bootstrap required, opened a new log
I20260810 18:55:09.201297 16427 ts_tablet_manager.cc:1397] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260810 18:55:09.202697 16431 raft_consensus.cc:482] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:55:09.202955 16431 raft_consensus.cc:504] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.204428 16431 leader_election.cc:288] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5766da1f194841829d266479b6e94f33 (127.30.153.129:44199), 5f5762c8d61b4e519d05df29e47a769b (127.30.153.130:33387)
I20260810 18:55:09.206154 16427 raft_consensus.cc:348] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.206491 16427 raft_consensus.cc:374] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:55:09.206559 16427 raft_consensus.cc:729] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5766da1f194841829d266479b6e94f33, State: Initialized, Role: FOLLOWER
I20260810 18:55:09.207113 16427 consensus_queue.cc:260] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.208300 16427 ts_tablet_manager.cc:1428] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 18:55:09.208686 15991 heartbeater.cc:503] Master 127.30.153.190:42097 was elected leader, sending a full tablet report...
I20260810 18:55:09.211196 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.204639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:44199 (local address 127.30.153.131:38133)
0810 18:55:09.205355 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.205370 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:09.205461 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:09.207491 (+ 2030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:09.207495 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:09.207518 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:55:09.207650 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.207657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.208634 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.208637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.209809 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.209820 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.210789 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.210793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.210893 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.210915 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:55:09.210983 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:55:09.211053 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635}
W20260810 18:55:09.211413 16200 tablet.cc:2369] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:55:09.211664 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.205885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:38133 (local address 127.30.153.129:44199)
0810 18:55:09.206307 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:09.206312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:09.206328 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:55:09.206376 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:09.206379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:09.206462 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:09.206539 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:55:09.207782 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.208501 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.209975 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.210566 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.210679 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.210729 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:09.211476 (+ 747us) server_negotiation.cc:299] Negotiation successful
0810 18:55:09.211559 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340}
I20260810 18:55:09.212225 15925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7390d6f13222420a8134c8c4101ebf50" candidate_uuid: "0ce5c17a57e24ab2831a24381d328b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5766da1f194841829d266479b6e94f33" is_pre_election: true
I20260810 18:55:09.212522 15925 raft_consensus.cc:2454] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ce5c17a57e24ab2831a24381d328b19 in term 0.
I20260810 18:55:09.212987 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.206013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:39593 (local address 127.30.153.130:33387)
0810 18:55:09.206718 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:09.206723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:09.208917 (+ 2194us) server_negotiation.cc:407] Connection header received
0810 18:55:09.209116 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:09.209120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:09.209170 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:09.209217 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:55:09.209623 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.210260 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.211716 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.212243 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.212349 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.212397 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:09.212789 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 18:55:09.212876 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618}
I20260810 18:55:09.213016 16229 leader_election.cc:302] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ce5c17a57e24ab2831a24381d328b19, 5766da1f194841829d266479b6e94f33; no voters:
I20260810 18:55:09.213332 16431 raft_consensus.cc:2781] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:55:09.213414 16435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.204904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:33387 (local address 127.30.153.131:39593)
0810 18:55:09.208806 (+ 3902us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.208823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:09.208988 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:09.209348 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:09.209350 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:09.209367 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:55:09.209472 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.209478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.210470 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.210473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.211563 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:09.211585 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.212514 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.212518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:09.212636 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.212660 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:55:09.213181 (+ 521us) client_negotiation.cc:240] Negotiation successful
0810 18:55:09.213259 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3831,"mutex_wait_us":1,"thread_start_us":154,"threads_started":1}
I20260810 18:55:09.213425 16431 raft_consensus.cc:482] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:55:09.213502 16431 raft_consensus.cc:3037] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:55:09.214754 16129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7390d6f13222420a8134c8c4101ebf50" candidate_uuid: "0ce5c17a57e24ab2831a24381d328b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f5762c8d61b4e519d05df29e47a769b" is_pre_election: true
I20260810 18:55:09.214954 16431 raft_consensus.cc:504] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.215061 16129 raft_consensus.cc:2454] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ce5c17a57e24ab2831a24381d328b19 in term 0.
I20260810 18:55:09.215348 16431 leader_election.cc:288] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [CANDIDATE]: Term 1 election: Requested vote from peers 5766da1f194841829d266479b6e94f33 (127.30.153.129:44199), 5f5762c8d61b4e519d05df29e47a769b (127.30.153.130:33387)
I20260810 18:55:09.216190 15925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7390d6f13222420a8134c8c4101ebf50" candidate_uuid: "0ce5c17a57e24ab2831a24381d328b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5766da1f194841829d266479b6e94f33"
I20260810 18:55:09.216315 15925 raft_consensus.cc:3037] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:55:09.217689 15925 raft_consensus.cc:2454] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ce5c17a57e24ab2831a24381d328b19 in term 1.
I20260810 18:55:09.218025 16229 leader_election.cc:302] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ce5c17a57e24ab2831a24381d328b19, 5766da1f194841829d266479b6e94f33; no voters:
I20260810 18:55:09.218115 16129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7390d6f13222420a8134c8c4101ebf50" candidate_uuid: "0ce5c17a57e24ab2831a24381d328b19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f5762c8d61b4e519d05df29e47a769b"
I20260810 18:55:09.218253 16129 raft_consensus.cc:3037] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:55:09.219386 16129 raft_consensus.cc:2454] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ce5c17a57e24ab2831a24381d328b19 in term 1.
I20260810 18:55:09.219429 16431 raft_consensus.cc:2781] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:55:09.219676 16431 raft_consensus.cc:686] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 1 LEADER]: Becoming Leader. State: Replica: 0ce5c17a57e24ab2831a24381d328b19, State: Running, Role: LEADER
I20260810 18:55:09.220088 16431 consensus_queue.cc:237] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } }
I20260810 18:55:09.224169 15677 catalog_manager.cc:5495] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ce5c17a57e24ab2831a24381d328b19 (127.30.153.131). New cstate: current_term: 1 leader_uuid: "0ce5c17a57e24ab2831a24381d328b19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ce5c17a57e24ab2831a24381d328b19" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37727 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 } health_report { overall_health: UNKNOWN } } }
Aug 10 18:55:09 dist-test-slave-68xk krb5kdc[14917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388108, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:09.256176 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.248706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.30.153.131:37727)
0810 18:55:09.249146 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:09.249152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:09.249166 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:55:09.249253 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:09.249256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:09.249322 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:09.249425 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:55:09.249860 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.250674 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.252641 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:09.252979 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:09.253104 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:09.253650 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:55:09.254298 (+ 648us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:55:09.254457 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:09.255937 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0810 18:55:09.256019 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340}
I20260810 18:55:09.271054 16129 raft_consensus.cc:1261] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 1 FOLLOWER]: Refusing update from remote peer 0ce5c17a57e24ab2831a24381d328b19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:55:09.272106 15925 raft_consensus.cc:1261] T 7390d6f13222420a8134c8c4101ebf50 P 5766da1f194841829d266479b6e94f33 [term 1 FOLLOWER]: Refusing update from remote peer 0ce5c17a57e24ab2831a24381d328b19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:55:09.273069 16438 consensus_queue.cc:1035] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f5762c8d61b4e519d05df29e47a769b" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 33387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:55:09.273262 16431 consensus_queue.cc:1035] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5766da1f194841829d266479b6e94f33" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 44199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:55:09.287889 16443 mvcc.cc:205] Tried to move back new op lower bound from 7317045695558160384 to 7317045695369003008. Current Snapshot: MvccSnapshot[applied={T|T < 7317045695558160384}]
I20260810 18:55:09.295766 16445 mvcc.cc:205] Tried to move back new op lower bound from 7317045695558160384 to 7317045695369003008. Current Snapshot: MvccSnapshot[applied={T|T < 7317045695558160384}]
I20260810 18:55:09.304682 16446 mvcc.cc:205] Tried to move back new op lower bound from 7317045695558160384 to 7317045695369003008. Current Snapshot: MvccSnapshot[applied={T|T < 7317045695558160384}]
I20260810 18:55:09.308346 15677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:55:09.308575 15677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:55:09.311631 15677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f45e215049c485bb5cce1b79519b02c: Sending DeleteTablet for 3 replicas of tablet 7390d6f13222420a8134c8c4101ebf50
I20260810 18:55:09.312855 16271 tablet_service.cc:1514] Processing DeleteTablet for tablet 7390d6f13222420a8134c8c4101ebf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:55:09 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:56760
I20260810 18:55:09.313323 15898 tablet_service.cc:1514] Processing DeleteTablet for tablet 7390d6f13222420a8134c8c4101ebf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:55:09 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:46728
I20260810 18:55:09.313603 16101 tablet_service.cc:1514] Processing DeleteTablet for tablet 7390d6f13222420a8134c8c4101ebf50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:55:09 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:42606
I20260810 18:55:09.314216 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 15793
I20260810 18:55:09.326694 16454 tablet_replica.cc:330] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: stopping tablet replica
I20260810 18:55:09.327109 16454 raft_consensus.cc:2229] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:55:09.327643 16454 raft_consensus.cc:2258] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:55:09.329424 16454 ts_tablet_manager.cc:1905] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:55:09.332549 16454 ts_tablet_manager.cc:1918] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:55:09.332679 16454 log.cc:1196] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/7390d6f13222420a8134c8c4101ebf50
I20260810 18:55:09.333067 16454 ts_tablet_manager.cc:1939] T 7390d6f13222420a8134c8c4101ebf50 P 0ce5c17a57e24ab2831a24381d328b19: Deleting consensus metadata
I20260810 18:55:09.334656 16457 tablet_replica.cc:330] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: stopping tablet replica
I20260810 18:55:09.334661 15656 catalog_manager.cc:4841] TS 0ce5c17a57e24ab2831a24381d328b19 (127.30.153.131:37727): tablet 7390d6f13222420a8134c8c4101ebf50 (table test-table [id=2d27d7205a7a428aa357fa7631c77718]) successfully deleted
I20260810 18:55:09.335041 16457 raft_consensus.cc:2229] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:55:09.335435 16457 raft_consensus.cc:2258] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:55:09.337191 16457 ts_tablet_manager.cc:1905] T 7390d6f13222420a8134c8c4101ebf50 P 5f5762c8d61b4e519d05df29e47a769b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 18:55:09.337651 15655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:44199 (error 108)
I20260810 18:55:09.339161 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 16002
I20260810 18:55:09.339218 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.338881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:44199 (local address 127.0.0.1:46738)
0810 18:55:09.339023 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.339117 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:44199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:55:09.341588 15655 catalog_manager.cc:4584] TS 5766da1f194841829d266479b6e94f33 (127.30.153.129:44199): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7390d6f13222420a8134c8c4101ebf50: Network error: Client connection negotiation failed: client connection to 127.30.153.129:44199: connect: Connection refused (error 111)
I20260810 18:55:09.374554 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.374122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:33387 (local address 127.0.0.1:42612)
0810 18:55:09.374320 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.374378 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.130:33387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260810 18:55:09.374688 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 16207
I20260810 18:55:09.394999 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.394642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:44199 (local address 127.0.0.1:46754)
0810 18:55:09.394841 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.394900 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:44199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260810 18:55:09.395980 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:09.395707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:44199 (local address 127.0.0.1:46764)
0810 18:55:09.395863 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:09.395913 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:44199: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260810 18:55:09.398838 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 15627
I20260810 18:55:09.421030 31334 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 18:55:09.770641 31334 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 18:55:09.770807 31334 mini_ranger.cc:66] Stopping Ranger...
I20260810 18:55:10.127074 31334 mini_ranger.cc:68] Stopped Ranger
2026-08-10 18:55:10.127 UTC [14965] LOG: received smart shutdown request
2026-08-10 18:55:10.131 UTC [14965] LOG: background worker "logical replication launcher" (PID 14981) exited with exit code 1
2026-08-10 18:55:10.132 UTC [14973] LOG: shutting down
2026-08-10 18:55:10.321 UTC [14965] LOG: database system is shut down
2026-08-10T18:55:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49839 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:55:10 dist-test-slave-68xk krb5kdc[16477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:55:10 dist-test-slave-68xk krb5kdc[16477](info): set up 2 sockets
Aug 10 18:55:10 dist-test-slave-68xk krb5kdc[16477](info): commencing operation
krb5kdc: starting...
W20260810 18:55:11.702989 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:55:11 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388111, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:55:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:55:11Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.153.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.153.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/rangeradmin_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/rangeradmin_127.30.153.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.153.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.153.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/rangerlookup_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/rangerlookup_127.30.153.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.153.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.153.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/HTTP_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/HTTP_127.30.153.148@KRBTEST.COM.keytab.
I20260810 18:55:11.894246 31334 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBp6AcV/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/postgres -l logfile start
2026-08-10 18:55:14.583 UTC [16520] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 18:55:14.583 UTC [16520] LOG: listening on IPv4 address "127.30.153.148", port 57533
2026-08-10 18:55:14.587 UTC [16520] LOG: listening on Unix socket "/tmp/.s.PGSQL.57533"
2026-08-10 18:55:14.609 UTC [16524] LOG: database system was shut down at 2026-08-10 18:55:13 UTC
2026-08-10 18:55:14.613 UTC [16520] LOG: database system is ready to accept connections
I20260810 18:55:15.883455 31334 mini_postgres.cc:96] Postgres bound to 57533
2026-08-10 18:55:15.891 UTC [16536] FATAL: database "slave" does not exist
127.30.153.148:57533 - accepting connections
I20260810 18:55:15.891919 31334 mini_ranger.cc:161] Starting Ranger...
I20260810 18:55:15.908519 31334 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 18:55:16.442922 31334 mini_ranger.cc:87] Created ranger Postgres database
I20260810 18:55:16.443022 31334 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-admin
2026-08-10 18:55:16,733 [I] DB FLAVOR :POSTGRES
2026-08-10 18:55:16,734 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:55:16,734 [I] Checking connection..
2026-08-10 18:55:16,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 18:55:16,980 [I] Checking connection passed.
2026-08-10 18:55:16,981 [I] --------- Verifying version history table ---------
2026-08-10 18:55:16,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:55:17,277 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:55:17,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:55:17,544 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 18:55:17,544 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 18:55:17,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 18:55:17.779 UTC [16633] WARNING: there is no transaction in progress
2026-08-10 18:55:17,787 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 18:55:17,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 18:55:18,054 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 18:55:18,054 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:55:18,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 18:55:18,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 18:55:18,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-68xk.c.gcp-upstream.internal','N') ;"
2026-08-10 18:55:18,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:55:19,110 [I] Table x_portal_user does not exist in database ranger
2026-08-10 18:55:19,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:55:19,374 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 18:55:19,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:55:19,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:55:19,900 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 18:55:19,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 18:55:20.922 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:20.939 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:20.957 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:20.974 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:20.991 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.010 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.071 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.078 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.094 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.105 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.116 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.130 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.141 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.148 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.155 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21.162 UTC [16797] WARNING: there is no transaction in progress
2026-08-10 18:55:21,879 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 18:55:21,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 18:55:22,116 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 18:55:22,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 18:55:22,383 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 18:55:22,383 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 18:55:22,658 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 18:55:22,658 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 18:55:22,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 18:55:23,190 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-68xk.c.gcp-upstream.internal';"
2026-08-10 18:55:23,442 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 18:55:23,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 18:55:23,706 [I] DB_PATCHES have already been applied
I20260810 18:55:23.712774 31334 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBp6AcV/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 18:55:23.712850 31334 mini_ranger.cc:193] Using host: 127.30.153.148
I20260810 18:55:23.717381 31334 mini_ranger.cc:227] Ranger admin URL: http://127.30.153.148:46451
Aug 10, 2026 6:55:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:55:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:55:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:55:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:55:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:55:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:55:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 18:55:25.080114 31334 mini_ranger.cc:160] Time spent starting Ranger: real 9.188s user 0.000s sys 0.013s
Aug 10, 2026 6:55:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:55:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 6:55:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:55:46 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:55:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 18:55:48.135968 31334 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.153.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.153.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/rangerkms_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.153.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/rangerkms_127.30.153.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.153.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.153.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.153.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/HTTP_127.30.153.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.153.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/HTTP_127.30.153.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 18:55:48 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388148, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 18:55:48.227557 31334 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 18:55:48.238489 31334 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 18:55:49.166728 31334 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 18:55:49.166817 31334 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms
I20260810 18:55:49.166831 31334 mini_ranger_kms.cc:227] Using postgres at 127.30.153.148:57533
2026-08-10 18:55:49,365 [I] DB FLAVOR :POSTGRES
2026-08-10 18:55:49,366 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 18:55:49,366 [I] Checking connection
2026-08-10 18:55:49,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 18:55:49,615 [I] connection success
2026-08-10 18:55:49,615 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 18:55:49,615 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 18:55:49,616 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 18:55:49,868 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 18:55:49,869 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 18:55:49,869 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 18:55:49,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.153.148:57533/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 18:55:50,306 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 18:55:51.183051 31334 mini_ranger_kms.cc:310] Created kms service
I20260810 18:55:51.392726 31334 mini_ranger_kms.cc:326] Created kudu user
I20260810 18:55:51.440586 31334 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 18:55:51 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388148, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:51.878878 31334 mini_ranger_kms.cc:384] Added ranger policy
I20260810 18:55:51.879160 31334 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBp6AcV/build/debug/bin/postgresql.jar:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBp6AcV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 18:55:51.879191 31334 mini_ranger_kms.cc:240] Using host: 127.30.153.148
I20260810 18:55:51.884578 31334 mini_ranger_kms.cc:276] Ranger KMS PID: 17072
I20260810 18:55:51.884690 31334 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.153.148:59421
Aug 10, 2026 6:55:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:55:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:55:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:55:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:55:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:55:53 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 18:55:54 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786388154, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.153.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:54.720499 31334 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.493s user 0.016s sys 0.169s
I20260810 18:55:54.720660 31334 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 18:55:54.720745 31334 mini_ranger_kms.cc:401] 127.30.153.148:59421/kms/v1/keys
Aug 10 18:55:54 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388154, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.153.148@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
Aug 10 18:55:55 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388154, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.153.148@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
Aug 10 18:55:57 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:55:57.705228 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:33485
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:33485
--ranger_config_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:59421/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:55:57.822543 17153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:55:57.822813 17153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:55:57.822854 17153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:55:57.826958 17153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:55:57.827015 17153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:55:57.827069 17153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:55:57.827100 17153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:55:57.827139 17153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:55:57.832826 17153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.153.148:59421/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:33485
--ranger_config_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:33485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:55:57.833940 17153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:55:57.835093 17153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:55:57.844561 17167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:57.844565 17163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:57.844561 17165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:57.844913 17153 server_base.cc:1034] running on GCE node
I20260810 18:55:57.845700 17153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:55:57.847177 17153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:55:57.848376 17153 hybrid_clock.cc:648] HybridClock initialized: now 1786388157848332 us; error 55 us; skew 500 ppm
Aug 10 18:55:57 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388157, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:57.857180 17153 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:55:57.858151 17153 webserver.cc:460] Webserver started at http://127.30.153.190:36127/ using document root <none> and password file <none>
I20260810 18:55:57.858803 17153 fs_manager.cc:359] Metadata directory not provided
I20260810 18:55:57.858861 17153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:55:57.859091 17153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:55:57 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388157, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:57.906620 17153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "1d015734a0d94e9090f6842531eb3a3b"
format_stamp: "Formatted at 2026-08-10 18:55:57 on dist-test-slave-68xk"
server_key: "2f72b64f6736dd0a26afb8adf6a4d778"
server_key_iv: "8140307fa76f9251af0aa81da7087012"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:57.907287 17153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "1d015734a0d94e9090f6842531eb3a3b"
format_stamp: "Formatted at 2026-08-10 18:55:57 on dist-test-slave-68xk"
server_key: "2f72b64f6736dd0a26afb8adf6a4d778"
server_key_iv: "8140307fa76f9251af0aa81da7087012"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:57.911520 17153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:55:57.926206 17179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:57.927703 17153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:55:57.927850 17153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "1d015734a0d94e9090f6842531eb3a3b"
format_stamp: "Formatted at 2026-08-10 18:55:57 on dist-test-slave-68xk"
server_key: "2f72b64f6736dd0a26afb8adf6a4d778"
server_key_iv: "8140307fa76f9251af0aa81da7087012"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:57.927994 17153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:55:57.944856 17153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:55:57.948172 17153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:55:57.948395 17153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:55:57.956869 17153 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:33485
I20260810 18:55:57.956885 17259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:33485 every 8 connection(s)
I20260810 18:55:57.957875 17153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:55:57.960796 17260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:55:57.963622 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 17153
I20260810 18:55:57.963769 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:55:57.964092 31334 external_mini_cluster.cc:1428] Setting key 05589c654d1cf7200c859287dc8efd52
I20260810 18:55:57.967376 17260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Bootstrap starting.
I20260810 18:55:57.970132 17260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Neither blocks nor log segments found. Creating new log.
I20260810 18:55:57.970961 17260 log.cc:824] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:55:57.973289 17260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: No bootstrap required, opened a new log
Aug 10 18:55:57 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:55:57.977710 17260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } }
I20260810 18:55:57.977988 17260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:55:57.978073 17260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d015734a0d94e9090f6842531eb3a3b, State: Initialized, Role: FOLLOWER
I20260810 18:55:57.978607 17260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } }
I20260810 18:55:57.978768 17260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:55:57.978853 17260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:55:57.978950 17260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:55:57.980050 17260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } }
I20260810 18:55:57.980311 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:57.965696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.30.153.190:33485)
0810 18:55:57.966187 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:57.966219 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:57.966264 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:55:57.967078 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:57.967102 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:57.967530 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:57.967888 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:57.968757 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:57.970210 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:57.971521 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:57.972081 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:57.972267 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:57.975280 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:57.975286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:57.975300 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:57.975327 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:57.976998 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:57.977467 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:57.977472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:57.977477 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:57.977580 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:57.977893 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:57.977897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:57.977899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:57.978449 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:57.978679 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:57.979311 (+ 632us) server_negotiation.cc:299] Negotiation successful
0810 18:55:57.979530 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":143,"threads_started":1}
I20260810 18:55:57.980408 17260 leader_election.cc:302] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d015734a0d94e9090f6842531eb3a3b; no voters:
I20260810 18:55:57.980798 17260 leader_election.cc:288] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:55:57.980883 17270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:55:57.981148 17270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 1 LEADER]: Becoming Leader. State: Replica: 1d015734a0d94e9090f6842531eb3a3b, State: Running, Role: LEADER
I20260810 18:55:57.981503 17270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } }
I20260810 18:55:57.982115 17260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:55:57.982798 17270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d015734a0d94e9090f6842531eb3a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } } }
I20260810 18:55:57.982991 17270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: This master's current role is: LEADER
I20260810 18:55:57.983265 17270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d015734a0d94e9090f6842531eb3a3b. Latest consensus state: current_term: 1 leader_uuid: "1d015734a0d94e9090f6842531eb3a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } } }
I20260810 18:55:57.983373 17270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: This master's current role is: LEADER
I20260810 18:55:57.983632 17284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:55:57.986948 17284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:55:57.987401 17260 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:55:57.999421 17284 catalog_manager.cc:1347] Generated new cluster ID: 7f87f9f53f1c4c1e83fa3a069c08c9f9
I20260810 18:55:57.999601 17284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:55:58.033221 17284 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:55:58.035712 17284 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:55:58.044566 17284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Generated new TSK 0
I20260810 18:55:58.046523 17284 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:55:58.774562 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:59421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33485
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:55:58.960143 17332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:55:58.960604 17332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:55:58.960744 17332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:55:58.967639 17332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:55:58.967787 17332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:55:58.967955 17332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:55:58.976936 17332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.153.148:59421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:55:58.978477 17332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:55:58.980013 17332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:55:58.993526 17342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:58.993741 17347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:58.993506 17343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:58.994014 17332 server_base.cc:1034] running on GCE node
I20260810 18:55:58.994663 17332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:55:58.995354 17332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:55:59.002772 17332 hybrid_clock.cc:648] HybridClock initialized: now 1786388159002726 us; error 53 us; skew 500 ppm
Aug 10 18:55:58 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388158, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:59.006647 17332 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:55:59.008000 17332 webserver.cc:460] Webserver started at http://127.30.153.129:40405/ using document root <none> and password file <none>
I20260810 18:55:59.008816 17332 fs_manager.cc:359] Metadata directory not provided
I20260810 18:55:59.008904 17332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:55:59.009158 17332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388158, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:59.026757 17332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "3544aaa9a1eb4e23a138c06bdb72551b"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "f3a6eb7d3dacb828d3d34ea673cdb068"
server_key_iv: "c8869c5a2d1b8a4693e67d4cbd41926d"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.027686 17332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "3544aaa9a1eb4e23a138c06bdb72551b"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "f3a6eb7d3dacb828d3d34ea673cdb068"
server_key_iv: "c8869c5a2d1b8a4693e67d4cbd41926d"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.033872 17332 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 18:55:59.055065 17356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.056887 17332 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:55:59.057024 17332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "3544aaa9a1eb4e23a138c06bdb72551b"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "f3a6eb7d3dacb828d3d34ea673cdb068"
server_key_iv: "c8869c5a2d1b8a4693e67d4cbd41926d"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.057171 17332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:55:59.081779 17332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:55:59.086107 17332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:55:59.086447 17332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:55:59.087412 17332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:55:59.088689 17332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:55:59.088766 17332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.088861 17332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:55:59.088896 17332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.099622 17332 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:46157
I20260810 18:55:59.100859 17332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:55:59.102965 17521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:46157 every 8 connection(s)
I20260810 18:55:59.106688 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 17332
I20260810 18:55:59.106817 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:55:59.107211 31334 external_mini_cluster.cc:1428] Setting key d98cc15717869202f9f9648c59e79a42
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388158, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:55:59.138156 17528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:59.106969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:53059 (local address 127.30.153.190:33485)
0810 18:55:59.114995 (+ 8026us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:59.115001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:59.115023 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:55:59.123511 (+ 8488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:59.123516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:59.123601 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:59.123724 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:59.125648 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.126664 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.128629 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.129029 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.129134 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:59.133090 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:59.133095 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:59.133098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:59.133142 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:59.135331 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:59.136137 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:59.136142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:59.136144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:59.136244 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:59.136709 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:59.136712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:59.136714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:59.136951 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:59.137029 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:59.137791 (+ 762us) server_negotiation.cc:299] Negotiation successful
0810 18:55:59.137937 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7923,"thread_start_us":84,"threads_started":1}
I20260810 18:55:59.140277 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:59.105794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33485 (local address 127.30.153.129:53059)
0810 18:55:59.107399 (+ 1605us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:59.107461 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:59.123123 (+ 15662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:59.124156 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:59.124168 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:59.124592 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:55:59.125429 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:59.125452 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.126836 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.126841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:59.128460 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:59.128472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.129391 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.129395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:59.129615 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:59.130467 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:55:59.130492 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:55:59.132858 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:55:59.135587 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:59.135594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:59.135612 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:59.135991 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:59.136376 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:59.136379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:59.136381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:59.136586 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:59.137133 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:55:59.137140 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:55:59.137485 (+ 345us) client_negotiation.cc:769] Sending connection context
0810 18:55:59.138698 (+ 1213us) client_negotiation.cc:240] Negotiation successful
0810 18:55:59.139095 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"thread_start_us":148,"threads_started":1}
I20260810 18:55:59.142150 17522 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33485
I20260810 18:55:59.142518 17522 heartbeater.cc:464] Registering TS with master...
I20260810 18:55:59.143184 17522 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
I20260810 18:55:59.145182 17201 ts_manager.cc:194] Registered new tserver with Master: 3544aaa9a1eb4e23a138c06bdb72551b (127.30.153.129:46157)
I20260810 18:55:59.147205 17201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:53059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:55:59.174137 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:59421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33485
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786388159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 18:55:59.343303 17532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:55:59.343786 17532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:55:59.343915 17532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:55:59.349761 17532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:55:59.349864 17532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:55:59.349982 17532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:55:59.356961 17532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.153.148:59421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:55:59.358243 17532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:55:59.359447 17532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:55:59.372797 17545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:59.379539 17540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:59.380786 17532 server_base.cc:1034] running on GCE node
W20260810 18:55:59.381291 17541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:59.382375 17532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:55:59.383325 17532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:55:59.401546 17532 hybrid_clock.cc:648] HybridClock initialized: now 1786388159401450 us; error 121 us; skew 500 ppm
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:59.406020 17532 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:55:59.407330 17532 webserver.cc:460] Webserver started at http://127.30.153.130:46859/ using document root <none> and password file <none>
I20260810 18:55:59.408178 17532 fs_manager.cc:359] Metadata directory not provided
I20260810 18:55:59.408352 17532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:55:59.408686 17532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:59.435137 17532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "64741c853ca84abc8a3dc8a57c5948db"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "c65a896e9157ebac21364a2dd34feb28"
server_key_iv: "6ea1204770934129023fce584fc052af"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.435948 17532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "64741c853ca84abc8a3dc8a57c5948db"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "c65a896e9157ebac21364a2dd34feb28"
server_key_iv: "6ea1204770934129023fce584fc052af"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.442225 17532 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 18:55:59.469229 17556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.471014 17532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:55:59.471155 17532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "64741c853ca84abc8a3dc8a57c5948db"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "c65a896e9157ebac21364a2dd34feb28"
server_key_iv: "6ea1204770934129023fce584fc052af"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.471289 17532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:55:59.495680 17532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:55:59.499840 17532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:55:59.500130 17532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:55:59.500952 17532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:55:59.502134 17532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:55:59.502249 17532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.502357 17532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:55:59.502460 17532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.518236 17532 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:44439
I20260810 18:55:59.518517 17730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:44439 every 8 connection(s)
I20260810 18:55:59.519647 17532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:55:59.526266 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 17532
I20260810 18:55:59.526434 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:55:59.526801 31334 external_mini_cluster.cc:1428] Setting key ec70a344bb7dc1860b1c6007f965c102
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:55:59.542269 17528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:59.523923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:47523 (local address 127.30.153.190:33485)
0810 18:55:59.526937 (+ 3014us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:59.526946 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:59.527670 (+ 724us) server_negotiation.cc:407] Connection header received
0810 18:55:59.528911 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:59.528915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:59.528972 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:59.529072 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:59.530867 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.531761 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.533753 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.534080 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.534204 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:59.537540 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:59.537543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:59.537546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:59.537575 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:59.539422 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:59.540157 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:59.540161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:59.540163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:59.540273 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:59.540759 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:59.540763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:59.540765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:59.541014 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:59.541126 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:59.541913 (+ 787us) server_negotiation.cc:299] Negotiation successful
0810 18:55:59.542072 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2882}
I20260810 18:55:59.544059 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:59.522773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33485 (local address 127.30.153.130:47523)
0810 18:55:59.527464 (+ 4691us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:59.527535 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:59.528684 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:59.529406 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:59.529417 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:59.529832 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:55:59.530646 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:59.530669 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.531980 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.531983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:59.533606 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:59.533619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.534297 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.534302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:59.534578 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:59.535350 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:55:59.535374 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:55:59.537334 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:55:59.539585 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:59.539598 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:59.539616 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:59.540008 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:59.540404 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:59.540407 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:59.540409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:59.540625 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:59.541216 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:55:59.541223 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:55:59.541587 (+ 364us) client_negotiation.cc:769] Sending connection context
0810 18:55:59.542502 (+ 915us) client_negotiation.cc:240] Negotiation successful
0810 18:55:59.542896 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4407,"thread_start_us":157,"threads_started":1}
I20260810 18:55:59.545570 17731 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33485
I20260810 18:55:59.545858 17731 heartbeater.cc:464] Registering TS with master...
I20260810 18:55:59.546608 17731 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:55:59.548044 17201 ts_manager.cc:194] Registered new tserver with Master: 64741c853ca84abc8a3dc8a57c5948db (127.30.153.130:44439)
I20260810 18:55:59.548913 17201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:47523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:55:59.578259 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:59421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33485
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:55:59.763922 17737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:55:59.764350 17737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:55:59.764484 17737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
W20260810 18:55:59.770548 17737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:55:59.770680 17737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:55:59.770828 17737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:55:59.779438 17737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.153.148:59421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:55:59.781361 17737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:55:59.783123 17737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:55:59.797447 17745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:59.798586 17748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:55:59.798827 17744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:55:59.799198 17737 server_base.cc:1034] running on GCE node
I20260810 18:55:59.799849 17737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:55:59.800702 17737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:55:59.816183 17737 hybrid_clock.cc:648] HybridClock initialized: now 1786388159816074 us; error 131 us; skew 500 ppm
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:55:59.820263 17737 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:55:59.821584 17737 webserver.cc:460] Webserver started at http://127.30.153.131:46023/ using document root <none> and password file <none>
I20260810 18:55:59.822510 17737 fs_manager.cc:359] Metadata directory not provided
I20260810 18:55:59.822683 17737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:55:59.823061 17737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:55:59.848716 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "d0be8282768c5152fe7d3fb9c09a9ecc"
server_key_iv: "fef4062b1ef8771a30bc59f1fc8be469"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.850651 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "d0be8282768c5152fe7d3fb9c09a9ecc"
server_key_iv: "fef4062b1ef8771a30bc59f1fc8be469"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.856470 17737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:55:59.891860 17756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.893824 17737 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260810 18:55:59.893996 17737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "d0be8282768c5152fe7d3fb9c09a9ecc"
server_key_iv: "fef4062b1ef8771a30bc59f1fc8be469"
server_key_version: "kuduclusterkey@0"
I20260810 18:55:59.894155 17737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:55:59.920074 17737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:55:59.927711 17737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:55:59.928020 17737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:55:59.928853 17737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:55:59.930060 17737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:55:59.930167 17737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.930267 17737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:55:59.930335 17737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:55:59.954389 17737 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:40055
I20260810 18:55:59.955928 17737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:55:59.958235 17928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:40055 every 8 connection(s)
I20260810 18:55:59.966943 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 17737
I20260810 18:55:59.967082 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:55:59.967417 31334 external_mini_cluster.cc:1428] Setting key fa94a8a85ca67b78d4571593eab0b4e6
Aug 10 18:55:59 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:55:59.977419 17528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:59.961941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:38105 (local address 127.30.153.190:33485)
0810 18:55:59.962057 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 18:55:59.962062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:55:59.963859 (+ 1797us) server_negotiation.cc:407] Connection header received
0810 18:55:59.965196 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:55:59.965200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:55:59.965259 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:55:59.965318 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:55:59.967008 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.967764 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.969622 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.969942 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.970027 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:59.973165 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:55:59.973169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:55:59.973172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:55:59.973199 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:55:59.974984 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:59.975623 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:59.975627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:59.975629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:59.975736 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:55:59.976159 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:55:59.976163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:55:59.976165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:55:59.976403 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:55:59.976494 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 18:55:59.977136 (+ 642us) server_negotiation.cc:299] Negotiation successful
0810 18:55:59.977259 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 18:55:59.979323 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:55:59.960831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33485 (local address 127.30.153.131:38105)
0810 18:55:59.963694 (+ 2863us) negotiation.cc:107] Waiting for socket to connect
0810 18:55:59.963752 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:55:59.964918 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:55:59.965756 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:55:59.965765 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:55:59.966124 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:55:59.966821 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:59.966843 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.967926 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.967929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:59.969486 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:55:59.969498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:55:59.970238 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:55:59.970242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:55:59.970477 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:55:59.971159 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:55:59.971181 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:55:59.972975 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:55:59.975130 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:59.975135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:59.975150 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:59.975494 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:59.975856 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:55:59.975859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:55:59.975861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:55:59.976041 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:55:59.976562 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:55:59.976568 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:55:59.976863 (+ 295us) client_negotiation.cc:769] Sending connection context
0810 18:55:59.977944 (+ 1081us) client_negotiation.cc:240] Negotiation successful
0810 18:55:59.978266 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2626,"thread_start_us":133,"threads_started":1}
I20260810 18:55:59.981891 17930 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33485
I20260810 18:55:59.982193 17930 heartbeater.cc:464] Registering TS with master...
I20260810 18:55:59.982985 17930 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
I20260810 18:55:59.984122 17201 ts_manager.cc:194] Registered new tserver with Master: b0d0a07f5e324ba4a7ec4e17cf9f2706 (127.30.153.131:40055)
I20260810 18:55:59.984872 17201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:38105
I20260810 18:55:59.993610 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:55:59.994693 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 17332
I20260810 18:56:00.015021 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 17532
I20260810 18:56:00.029208 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 17737
I20260810 18:56:00.051035 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 17153
I20260810 18:56:00.080637 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:33485
--webserver_interface=127.30.153.190
--webserver_port=36127
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:33485
--ranger_config_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:59421/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.153.148:59421/kms with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:00.196020 17933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:00.196322 17933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:00.196404 17933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:00.201124 17933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:00.201215 17933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:00.201253 17933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:00.201292 17933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:00.201330 17933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:00.206763 17933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.153.148:59421/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:33485
--ranger_config_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:33485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=36127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:00.207782 17933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:00.208806 17933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:00.217324 17949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:00.217396 17947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:00.217384 17945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:00.217659 17933 server_base.cc:1034] running on GCE node
I20260810 18:56:00.218240 17933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:00.219376 17933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:00.220547 17933 hybrid_clock.cc:648] HybridClock initialized: now 1786388160220530 us; error 32 us; skew 500 ppm
Aug 10 18:56:00 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:00.223543 17933 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:56:00.224466 17933 webserver.cc:460] Webserver started at http://127.30.153.190:36127/ using document root <none> and password file <none>
I20260810 18:56:00.225015 17933 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:00.225140 17933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:56:00 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:56:00.307698 17933 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:56:00.310060 17961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:00.311492 17933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:56:00.311650 17933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "1d015734a0d94e9090f6842531eb3a3b"
format_stamp: "Formatted at 2026-08-10 18:55:57 on dist-test-slave-68xk"
server_key: "2f72b64f6736dd0a26afb8adf6a4d778"
server_key_iv: "8140307fa76f9251af0aa81da7087012"
server_key_version: "kuduclusterkey@0"
I20260810 18:56:00.312493 17933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:00.335109 17933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:00.338318 17933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:00.338562 17933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:00.346438 17933 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:33485
I20260810 18:56:00.346503 18041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:33485 every 8 connection(s)
I20260810 18:56:00.347599 17933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:56:00.351387 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 17933
I20260810 18:56:00.351928 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:59421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:46157
--local_ip_for_outbound_sockets=127.30.153.129
--tserver_master_addrs=127.30.153.190:33485
--webserver_port=40405
--webserver_interface=127.30.153.129
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.153.148:59421/kms with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:00.352285 18042 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:56:00.354763 18042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Bootstrap starting.
I20260810 18:56:00.371737 18042 log.cc:824] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:00.378713 18042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:56:00.379120 18042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Bootstrap complete.
I20260810 18:56:00.383298 18042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } }
I20260810 18:56:00.383690 18042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d015734a0d94e9090f6842531eb3a3b, State: Initialized, Role: FOLLOWER
I20260810 18:56:00.384215 18042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } }
I20260810 18:56:00.384361 18042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:56:00.384449 18042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:56:00.384567 18042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:56:00.386643 18042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } }
I20260810 18:56:00.387025 18042 leader_election.cc:302] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d015734a0d94e9090f6842531eb3a3b; no voters:
I20260810 18:56:00.387398 18042 leader_election.cc:288] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:56:00.387485 18048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:56:00.387758 18048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [term 2 LEADER]: Becoming Leader. State: Replica: 1d015734a0d94e9090f6842531eb3a3b, State: Running, Role: LEADER
I20260810 18:56:00.388108 18048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } }
I20260810 18:56:00.388652 18042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:00.389487 18050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1d015734a0d94e9090f6842531eb3a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } } }
I20260810 18:56:00.389626 18050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:00.389958 18051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d015734a0d94e9090f6842531eb3a3b. Latest consensus state: current_term: 2 leader_uuid: "1d015734a0d94e9090f6842531eb3a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d015734a0d94e9090f6842531eb3a3b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33485 } } }
I20260810 18:56:00.390100 18051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:00.390455 18061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:56:00.393630 18061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:56:00.394145 18042 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 18:56:00.394371 18061 catalog_manager.cc:1259] Loaded cluster ID: 7f87f9f53f1c4c1e83fa3a069c08c9f9
I20260810 18:56:00.394474 18061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:56:00.399748 18061 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:56:00.400866 18061 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Loaded TSK: 0
I20260810 18:56:00.402300 18061 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:56:00.495308 18046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:00.495645 18046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:00.495723 18046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:00.501653 18046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:00.501791 18046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:00.501909 18046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:56:00.509533 18046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.153.148:59421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:46157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=40405
--tserver_master_addrs=127.30.153.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:00.510888 18046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:00.512183 18046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:00.524869 18102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:00.524870 18101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:00.525416 18046 server_base.cc:1034] running on GCE node
W20260810 18:56:00.525489 18105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:00.525938 18046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:00.526760 18046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:00.530040 18046 hybrid_clock.cc:648] HybridClock initialized: now 1786388160530000 us; error 55 us; skew 500 ppm
Aug 10 18:56:00 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:00.533784 18046 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:56:00.535001 18046 webserver.cc:460] Webserver started at http://127.30.153.129:40405/ using document root <none> and password file <none>
I20260810 18:56:00.535861 18046 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:00.536011 18046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:56:00 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:56:00.574798 18046 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.004s sys 0.000s
I20260810 18:56:00.587117 18116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:00.589103 18046 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.000s
I20260810 18:56:00.589274 18046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "3544aaa9a1eb4e23a138c06bdb72551b"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "f3a6eb7d3dacb828d3d34ea673cdb068"
server_key_iv: "c8869c5a2d1b8a4693e67d4cbd41926d"
server_key_version: "kuduclusterkey@0"
I20260810 18:56:00.590094 18046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:00.630771 18046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:00.634591 18046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:00.634904 18046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:00.635656 18046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:00.636746 18046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:00.636854 18046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:00.636983 18046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:00.637066 18046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:00.648749 18046 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:46157
I20260810 18:56:00.649034 18292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:46157 every 8 connection(s)
I20260810 18:56:00.650198 18046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:56:00.654580 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 18046
I20260810 18:56:00.655172 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:59421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:44439
--local_ip_for_outbound_sockets=127.30.153.130
--tserver_master_addrs=127.30.153.190:33485
--webserver_port=46859
--webserver_interface=127.30.153.130
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.153.148:59421/kms with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:56:00 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:00.673098 18296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:00.652610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:53519 (local address 127.30.153.190:33485)
0810 18:56:00.655461 (+ 2851us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:00.655484 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:00.655533 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:56:00.655810 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:00.655821 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:00.656121 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:00.656463 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:00.657836 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:00.659215 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:00.661013 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:00.661796 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:00.661992 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:00.665332 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:00.665339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:00.665360 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:00.665396 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:00.667785 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:00.668551 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:00.668559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:00.668564 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:00.668719 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:00.669187 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:00.669193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:00.669198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:00.670111 (+ 913us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:00.670269 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:00.671827 (+ 1558us) server_negotiation.cc:299] Negotiation successful
0810 18:56:00.672113 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2665,"thread_start_us":122,"threads_started":1}
I20260810 18:56:00.674099 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:00.652746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33485 (local address 127.30.153.129:53519)
0810 18:56:00.653751 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:00.653809 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:00.654892 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:00.656535 (+ 1643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:00.656542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:00.656903 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:00.657596 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:00.657620 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:00.659368 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:00.659375 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:00.660849 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:00.660864 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:00.662007 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:00.662014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:00.662269 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:00.663052 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:00.663082 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:00.665061 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:00.667951 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:00.667958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:00.667973 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:00.668386 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:00.668841 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:00.668845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:00.668847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:00.669052 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:00.670264 (+ 1212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:00.670271 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:00.670613 (+ 342us) client_negotiation.cc:769] Sending connection context
0810 18:56:00.670824 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 18:56:00.671162 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"thread_start_us":142,"threads_started":1}
I20260810 18:56:00.677384 18294 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33485
I20260810 18:56:00.677624 18294 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:00.678186 18294 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
W20260810 18:56:00.680559 18294 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:33485 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:56:00.680682 18294 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:00.680852 18294 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
I20260810 18:56:00.681923 18294 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:00.682092 18294 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
W20260810 18:56:00.682934 18294 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:56:00.799808 18299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:00.800233 18299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:00.800338 18299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:00.806452 18299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:00.806588 18299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:00.806715 18299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:56:00.815537 18299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.153.148:59421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:44439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=46859
--tserver_master_addrs=127.30.153.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:00.817147 18299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:00.818761 18299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:00.833128 18309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:00.834324 18305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:00.834841 18299 server_base.cc:1034] running on GCE node
W20260810 18:56:00.835153 18304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:00.835734 18299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:00.836611 18299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:00.837824 18299 hybrid_clock.cc:648] HybridClock initialized: now 1786388160837795 us; error 44 us; skew 500 ppm
Aug 10 18:56:00 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:00.841933 18299 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:56:00.843130 18299 webserver.cc:460] Webserver started at http://127.30.153.130:46859/ using document root <none> and password file <none>
I20260810 18:56:00.843888 18299 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:00.844139 18299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:56:00 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:56:00.866542 18299 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:56:00.873332 18320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:00.875258 18299 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260810 18:56:00.875406 18299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "64741c853ca84abc8a3dc8a57c5948db"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "c65a896e9157ebac21364a2dd34feb28"
server_key_iv: "6ea1204770934129023fce584fc052af"
server_key_version: "kuduclusterkey@0"
I20260810 18:56:00.876173 18299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:00.909446 18299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:00.913159 18299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:00.913475 18299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:00.914336 18299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:00.915547 18299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:00.915659 18299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:00.915758 18299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:00.915822 18299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:00.928787 18299 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:44439
I20260810 18:56:00.928905 18477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:44439 every 8 connection(s)
I20260810 18:56:00.930136 18299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:56:00.939437 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 18299
I20260810 18:56:00.939905 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.153.148:59421/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:40055
--local_ip_for_outbound_sockets=127.30.153.131
--tserver_master_addrs=127.30.153.190:33485
--webserver_port=46023
--webserver_interface=127.30.153.131
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.153.148:59421/kms with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:56:00 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:00.947959 18296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:00.932596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:58451 (local address 127.30.153.190:33485)
0810 18:56:00.932770 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:00.932775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:00.933738 (+ 963us) server_negotiation.cc:407] Connection header received
0810 18:56:00.934877 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:00.934881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:00.934945 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:00.935029 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:00.936165 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:00.936712 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:00.938038 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:00.938349 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:00.938488 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:00.943146 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:00.943152 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:00.943158 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:00.943195 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:00.945119 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:00.945833 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:00.945837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:00.945839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:00.945935 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:00.946389 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:00.946393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:00.946395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:00.946685 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:00.946768 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:00.947628 (+ 860us) server_negotiation.cc:299] Negotiation successful
0810 18:56:00.947782 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:56:00.949965 18480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:00.933049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33485 (local address 127.30.153.130:58451)
0810 18:56:00.933557 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:00.933612 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:00.934653 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:00.935216 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:00.935224 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:00.935506 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:00.935998 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:00.936012 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:00.936849 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:00.936852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:00.937900 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:00.937910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:00.938563 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:00.938571 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:00.938868 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:00.939608 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:00.939630 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:00.942894 (+ 3264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:00.945287 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:00.945294 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:00.945313 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:00.945694 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:00.946059 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:00.946062 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:00.946064 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:00.946266 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:00.946891 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:00.946898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:00.947290 (+ 392us) client_negotiation.cc:769] Sending connection context
0810 18:56:00.948422 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0810 18:56:00.948813 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":148,"threads_started":1}
I20260810 18:56:00.952018 18478 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33485
I20260810 18:56:00.952374 18478 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:00.953104 18478 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
W20260810 18:56:00.955080 18478 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:33485 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:56:00.955233 18478 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:00.955435 18478 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
I20260810 18:56:00.956270 18478 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:00.956467 18478 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
W20260810 18:56:00.957294 18478 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:56:01.143692 18481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:01.144332 18481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:01.144523 18481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:01.155401 18481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:01.155573 18481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:01.155741 18481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:56:01.169661 18481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.153.148:59421/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:40055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=46023
--tserver_master_addrs=127.30.153.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:01.171593 18481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:01.173278 18481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:01.187474 18500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:01.187599 18503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:01.188097 18499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:01.188501 18481 server_base.cc:1034] running on GCE node
I20260810 18:56:01.189251 18481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:01.190214 18481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:01.193615 18481 hybrid_clock.cc:648] HybridClock initialized: now 1786388161193551 us; error 66 us; skew 500 ppm
Aug 10 18:56:01 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:01.198185 18481 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:56:01.199647 18481 webserver.cc:460] Webserver started at http://127.30.153.131:46023/ using document root <none> and password file <none>
I20260810 18:56:01.200515 18481 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:01.200639 18481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 18:56:01 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:56:01.232717 18481 fs_manager.cc:709] Time spent opening directory manager: real 0.010s user 0.002s sys 0.000s
I20260810 18:56:01.239066 18514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:01.240861 18481 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260810 18:56:01.241088 18481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706"
format_stamp: "Formatted at 2026-08-10 18:55:59 on dist-test-slave-68xk"
server_key: "d0be8282768c5152fe7d3fb9c09a9ecc"
server_key_iv: "fef4062b1ef8771a30bc59f1fc8be469"
server_key_version: "kuduclusterkey@0"
I20260810 18:56:01.241941 18481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:01.281185 18481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:01.285372 18481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:01.285663 18481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:01.286520 18481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:01.287806 18481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:01.287935 18481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:01.288055 18481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:01.288146 18481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:01.300561 18481 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:40055
I20260810 18:56:01.303534 18481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:56:01.306010 18668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:40055 every 8 connection(s)
I20260810 18:56:01.312883 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 18481
Aug 10 18:56:01 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:01.332465 18296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.309547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:59207 (local address 127.30.153.190:33485)
0810 18:56:01.309698 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:01.309703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:01.310274 (+ 571us) server_negotiation.cc:407] Connection header received
0810 18:56:01.311650 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:01.311655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:01.311710 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:01.311772 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:01.314045 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:01.314915 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:01.316655 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:01.316981 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:01.317072 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:01.321547 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:01.321551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:01.321554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:01.321610 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:01.325183 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:01.330339 (+ 5156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:01.330343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:01.330345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:01.330500 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:01.331000 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:01.331004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:01.331005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:01.331260 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:01.331342 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:01.332135 (+ 793us) server_negotiation.cc:299] Negotiation successful
0810 18:56:01.332284 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:56:01.334324 18672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.308338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33485 (local address 127.30.153.131:59207)
0810 18:56:01.310078 (+ 1740us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:01.310152 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:01.311365 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:01.312206 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:01.312216 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:01.312611 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:01.313861 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:01.313892 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:01.315089 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:01.315092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:01.316480 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:01.316491 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:01.317290 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:01.317294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:01.317495 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:01.318228 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:01.318251 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:01.321288 (+ 3037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:01.329797 (+ 8509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:01.329804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:01.329824 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:01.330208 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:01.330653 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:01.330657 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:01.330659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:01.330873 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:01.331430 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:01.331437 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:01.331816 (+ 379us) client_negotiation.cc:769] Sending connection context
0810 18:56:01.332783 (+ 967us) client_negotiation.cc:240] Negotiation successful
0810 18:56:01.333157 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1491,"thread_start_us":168,"threads_started":1}
I20260810 18:56:01.336078 18669 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33485
I20260810 18:56:01.336349 18669 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:01.336987 18669 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
I20260810 18:56:01.337184 18675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.313597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33788 (local address 127.30.153.190:33485)
0810 18:56:01.314633 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:01.314641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:01.314656 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:56:01.314694 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:01.314697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:01.314740 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:01.315839 (+ 1099us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:01.315936 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:01.316802 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:01.318135 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:01.318507 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:01.319003 (+ 496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:01.327261 (+ 8258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:01.327264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:01.327266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:01.327289 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:01.334288 (+ 6999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:01.334938 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:01.334941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:01.334943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:01.335032 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:01.335529 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:01.335532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:01.335534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:01.335767 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:01.335971 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:01.336867 (+ 896us) server_negotiation.cc:299] Negotiation successful
0810 18:56:01.336997 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":953,"thread_start_us":124,"threads_started":1}
I20260810 18:56:01.338604 17988 ts_manager.cc:194] Registered new tserver with Master: b0d0a07f5e324ba4a7ec4e17cf9f2706 (127.30.153.131:40055)
I20260810 18:56:01.340202 17988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:59207
Aug 10 18:56:01 dist-test-slave-68xk krb5kdc[16477](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786388161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:01.683174 18294 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:01.683579 18294 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
I20260810 18:56:01.684332 17987 ts_manager.cc:194] Registered new tserver with Master: 3544aaa9a1eb4e23a138c06bdb72551b (127.30.153.129:46157)
I20260810 18:56:01.685166 17987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:53519
I20260810 18:56:01.957563 18478 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:01.957887 18478 heartbeater.cc:511] Master 127.30.153.190:33485 requested a full tablet report, sending...
I20260810 18:56:01.958758 17987 ts_manager.cc:194] Registered new tserver with Master: 64741c853ca84abc8a3dc8a57c5948db (127.30.153.130:44439)
I20260810 18:56:01.959566 17987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:58451
I20260810 18:56:01.963587 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:56:01.974303 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.965261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.30.153.190:33485)
0810 18:56:01.965498 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:01.965503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:01.965655 (+ 152us) server_negotiation.cc:407] Connection header received
0810 18:56:01.965823 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:01.965826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:01.965881 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:01.965975 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:01.966767 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:01.967281 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:01.968147 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:01.968400 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:01.968515 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:01.970160 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:01.970164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:01.970166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:01.970194 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:01.972046 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:01.972564 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:01.972567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:01.972569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:01.972720 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:01.973018 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:01.973021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:01.973022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:01.973230 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:01.973350 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:01.974004 (+ 654us) server_negotiation.cc:299] Negotiation successful
0810 18:56:01.974138 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":65,"threads_started":1}
I20260810 18:56:01.979771 17987 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:56:01.984524 17987 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 10 18:56:01 dist-test-slave-68xk krb5kdc[16477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388161, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for HTTP/127.30.153.148@KRBTEST.COM
I20260810 18:56:02.016160 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.997957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:40055 (local address 127.0.0.1:47550)
0810 18:56:02.003798 (+ 5841us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:02.003812 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:02.003904 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:02.006487 (+ 2583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:02.006490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:02.006510 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:02.007460 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.007522 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.009177 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.009180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.012355 (+ 3175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.012370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.015568 (+ 3198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.015572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.015699 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.015745 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 18:56:02.015857 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 18:56:02.015993 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5751,"thread_start_us":116,"threads_started":1}
I20260810 18:56:02.016793 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.998746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.30.153.131:40055)
0810 18:56:02.004429 (+ 5683us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:02.004437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:02.004468 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:56:02.004548 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:02.004553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:02.004775 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:02.004931 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:02.007880 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.009038 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.012508 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.013335 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.013460 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.013602 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:02.016537 (+ 2935us) server_negotiation.cc:299] Negotiation successful
0810 18:56:02.016657 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":77,"threads_started":1}
I20260810 18:56:02.018266 18702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.996709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37330 (local address 127.30.153.130:44439)
0810 18:56:01.999010 (+ 2301us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:01.999017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:01.999040 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:56:01.999122 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:01.999128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:01.999351 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:02.000405 (+ 1054us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:02.002912 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.003992 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.008329 (+ 4337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.016815 (+ 8486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.017746 (+ 931us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.017918 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:02.018002 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 18:56:02.018108 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2186,"thread_start_us":85,"threads_started":1}
I20260810 18:56:02.017622 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.996587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:44439 (local address 127.0.0.1:37330)
0810 18:56:01.998513 (+ 1926us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:01.998569 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:01.998714 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:01.999533 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:01.999539 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:01.999566 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:02.002732 (+ 3166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.002740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.006043 (+ 3303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.006047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.008179 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.008188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.017034 (+ 8846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.017040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.017157 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.017184 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:56:02.017245 (+ 61us) client_negotiation.cc:240] Negotiation successful
0810 18:56:02.017326 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1714,"mutex_wait_us":41,"thread_start_us":125,"threads_started":1}
I20260810 18:56:02.021195 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.998328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46157 (local address 127.0.0.1:60444)
0810 18:56:02.004211 (+ 5883us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:02.004225 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:02.004318 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:02.008082 (+ 3764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:02.008085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:02.008106 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:02.008381 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.008386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.009908 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.009912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.011024 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.011036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.020826 (+ 9790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.020830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.020920 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.020944 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:56:02.020992 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 18:56:02.021062 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5818,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260810 18:56:02.022070 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:01.998524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.30.153.129:46157)
0810 18:56:02.004153 (+ 5629us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:02.004161 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:02.006047 (+ 1886us) server_negotiation.cc:407] Connection header received
0810 18:56:02.006146 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:02.006168 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:02.006374 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:02.008532 (+ 2158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:02.008679 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.009751 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.019229 (+ 9478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.020054 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.021636 (+ 1582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.021787 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:02.021841 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 18:56:02.021936 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5514,"thread_start_us":71,"threads_started":1}
I20260810 18:56:02.024164 18380 tablet_service.cc:1467] Processing CreateTablet for tablet 91512b5e38de454ea931f2a9a739a86b (DEFAULT_TABLE table=test-table [id=19a7e8df8ff146018b9f6235af294b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:02.025554 18380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91512b5e38de454ea931f2a9a739a86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:02.030187 18576 tablet_service.cc:1467] Processing CreateTablet for tablet 91512b5e38de454ea931f2a9a739a86b (DEFAULT_TABLE table=test-table [id=19a7e8df8ff146018b9f6235af294b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:02.031365 18576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91512b5e38de454ea931f2a9a739a86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:02.038275 18197 tablet_service.cc:1467] Processing CreateTablet for tablet 91512b5e38de454ea931f2a9a739a86b (DEFAULT_TABLE table=test-table [id=19a7e8df8ff146018b9f6235af294b29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:02.039546 18197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91512b5e38de454ea931f2a9a739a86b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:02.049715 18710 tablet_bootstrap.cc:492] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b: Bootstrap starting.
I20260810 18:56:02.053568 18710 tablet_bootstrap.cc:654] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:02.054699 18710 log.cc:824] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:02.065955 18710 tablet_bootstrap.cc:492] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b: No bootstrap required, opened a new log
I20260810 18:56:02.066469 18712 tablet_bootstrap.cc:492] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: Bootstrap starting.
I20260810 18:56:02.069592 18712 tablet_bootstrap.cc:654] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:02.070343 18713 tablet_bootstrap.cc:492] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: Bootstrap starting.
I20260810 18:56:02.070807 18712 log.cc:824] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:02.072868 18713 tablet_bootstrap.cc:654] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:02.073647 18712 tablet_bootstrap.cc:492] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: No bootstrap required, opened a new log
I20260810 18:56:02.073720 18713 log.cc:824] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:02.073855 18712 ts_tablet_manager.cc:1397] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 18:56:02.076810 18713 tablet_bootstrap.cc:492] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: No bootstrap required, opened a new log
I20260810 18:56:02.077078 18713 ts_tablet_manager.cc:1397] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20260810 18:56:02.078673 18710 ts_tablet_manager.cc:1397] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.003s
I20260810 18:56:02.078482 18712 raft_consensus.cc:348] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.078771 18712 raft_consensus.cc:374] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:02.078859 18712 raft_consensus.cc:729] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0d0a07f5e324ba4a7ec4e17cf9f2706, State: Initialized, Role: FOLLOWER
I20260810 18:56:02.079501 18712 consensus_queue.cc:260] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.081054 18712 ts_tablet_manager.cc:1428] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:56:02.081501 18669 heartbeater.cc:503] Master 127.30.153.190:33485 was elected leader, sending a full tablet report...
I20260810 18:56:02.082356 18713 raft_consensus.cc:348] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.082679 18713 raft_consensus.cc:374] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:02.082736 18713 raft_consensus.cc:729] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64741c853ca84abc8a3dc8a57c5948db, State: Initialized, Role: FOLLOWER
I20260810 18:56:02.083243 18713 consensus_queue.cc:260] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.083230 18710 raft_consensus.cc:348] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.083565 18710 raft_consensus.cc:374] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:02.083662 18710 raft_consensus.cc:729] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3544aaa9a1eb4e23a138c06bdb72551b, State: Initialized, Role: FOLLOWER
I20260810 18:56:02.084178 18710 consensus_queue.cc:260] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.084343 18713 ts_tablet_manager.cc:1428] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:56:02.084733 18478 heartbeater.cc:503] Master 127.30.153.190:33485 was elected leader, sending a full tablet report...
I20260810 18:56:02.085345 18710 ts_tablet_manager.cc:1428] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260810 18:56:02.085951 18294 heartbeater.cc:503] Master 127.30.153.190:33485 was elected leader, sending a full tablet report...
I20260810 18:56:02.124495 18717 raft_consensus.cc:482] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:02.124799 18717 raft_consensus.cc:504] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.126233 18717 leader_election.cc:288] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0d0a07f5e324ba4a7ec4e17cf9f2706 (127.30.153.131:40055), 3544aaa9a1eb4e23a138c06bdb72551b (127.30.153.129:46157)
I20260810 18:56:02.131665 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:02.126659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:58865 (local address 127.30.153.131:40055)
0810 18:56:02.126814 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:02.126819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:02.127115 (+ 296us) server_negotiation.cc:407] Connection header received
0810 18:56:02.127449 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:02.127453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:02.127499 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:02.127547 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:02.127976 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.128741 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.130320 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.130945 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.131041 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.131095 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:02.131431 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 18:56:02.131514 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:56:02.131948 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:02.127265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:60475 (local address 127.30.153.129:46157)
0810 18:56:02.127396 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:02.127400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:02.127522 (+ 122us) server_negotiation.cc:407] Connection header received
0810 18:56:02.127711 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:02.127715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:02.127757 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:02.127801 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:02.128186 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.128954 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.130625 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.131240 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.131338 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.131384 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:02.131745 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 18:56:02.131826 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:56:02.132545 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:02.126489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:40055 (local address 127.30.153.130:58865)
0810 18:56:02.127015 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:02.127031 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:02.127200 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:02.127700 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:02.127703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:02.127720 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:02.127859 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.127864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.128886 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.128889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.130173 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.130185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.131192 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.131195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.131292 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.131315 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:56:02.132165 (+ 850us) client_negotiation.cc:240] Negotiation successful
0810 18:56:02.132336 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":54,"thread_start_us":142,"threads_started":1}
I20260810 18:56:02.132552 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:02.126911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46157 (local address 127.30.153.130:60475)
0810 18:56:02.127442 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:02.127455 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:02.127597 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:02.127925 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:02.127928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:02.127943 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:02.128059 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.128065 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.129104 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.129107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.130465 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:02.130476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.131480 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.131484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:02.131603 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.131623 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:56:02.132231 (+ 608us) client_negotiation.cc:240] Negotiation successful
0810 18:56:02.132312 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":53,"thread_start_us":87,"threads_started":1}
I20260810 18:56:02.133543 18582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91512b5e38de454ea931f2a9a739a86b" candidate_uuid: "64741c853ca84abc8a3dc8a57c5948db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" is_pre_election: true
I20260810 18:56:02.133787 18207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91512b5e38de454ea931f2a9a739a86b" candidate_uuid: "64741c853ca84abc8a3dc8a57c5948db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" is_pre_election: true
I20260810 18:56:02.133932 18582 raft_consensus.cc:2454] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64741c853ca84abc8a3dc8a57c5948db in term 0.
I20260810 18:56:02.134081 18207 raft_consensus.cc:2454] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64741c853ca84abc8a3dc8a57c5948db in term 0.
I20260810 18:56:02.134588 18326 leader_election.cc:302] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64741c853ca84abc8a3dc8a57c5948db, b0d0a07f5e324ba4a7ec4e17cf9f2706; no voters:
I20260810 18:56:02.134963 18717 raft_consensus.cc:2781] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:56:02.135064 18717 raft_consensus.cc:482] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:56:02.135105 18717 raft_consensus.cc:3037] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:02.136719 18717 raft_consensus.cc:504] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.137120 18717 leader_election.cc:288] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [CANDIDATE]: Term 1 election: Requested vote from peers b0d0a07f5e324ba4a7ec4e17cf9f2706 (127.30.153.131:40055), 3544aaa9a1eb4e23a138c06bdb72551b (127.30.153.129:46157)
I20260810 18:56:02.137698 18582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91512b5e38de454ea931f2a9a739a86b" candidate_uuid: "64741c853ca84abc8a3dc8a57c5948db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706"
I20260810 18:56:02.137717 18207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91512b5e38de454ea931f2a9a739a86b" candidate_uuid: "64741c853ca84abc8a3dc8a57c5948db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3544aaa9a1eb4e23a138c06bdb72551b"
I20260810 18:56:02.137840 18207 raft_consensus.cc:3037] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:02.137846 18582 raft_consensus.cc:3037] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:02.139184 18207 raft_consensus.cc:2454] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64741c853ca84abc8a3dc8a57c5948db in term 1.
I20260810 18:56:02.139456 18582 raft_consensus.cc:2454] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64741c853ca84abc8a3dc8a57c5948db in term 1.
I20260810 18:56:02.139832 18326 leader_election.cc:302] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64741c853ca84abc8a3dc8a57c5948db, b0d0a07f5e324ba4a7ec4e17cf9f2706; no voters:
I20260810 18:56:02.140050 18717 raft_consensus.cc:2781] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:02.140326 18717 raft_consensus.cc:686] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 1 LEADER]: Becoming Leader. State: Replica: 64741c853ca84abc8a3dc8a57c5948db, State: Running, Role: LEADER
I20260810 18:56:02.140682 18717 consensus_queue.cc:237] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } }
I20260810 18:56:02.144330 17988 catalog_manager.cc:5495] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db reported cstate change: term changed from 0 to 1, leader changed from <none> to 64741c853ca84abc8a3dc8a57c5948db (127.30.153.130). New cstate: current_term: 1 leader_uuid: "64741c853ca84abc8a3dc8a57c5948db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64741c853ca84abc8a3dc8a57c5948db" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 } health_report { overall_health: UNKNOWN } } }
W20260810 18:56:02.152663 18295 tablet.cc:2369] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:56:02.163403 18702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:02.159788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37338 (local address 127.30.153.130:44439)
0810 18:56:02.159933 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:02.159938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:02.159951 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:56:02.160032 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:02.160034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:02.160094 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:02.160211 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:56:02.160479 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.160989 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.161879 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:02.162110 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:02.162222 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:02.162586 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:56:02.162977 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:56:02.163123 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:02.163185 (+ 62us) server_negotiation.cc:299] Negotiation successful
0810 18:56:02.163240 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:56:02.169395 18207 raft_consensus.cc:1261] T 91512b5e38de454ea931f2a9a739a86b P 3544aaa9a1eb4e23a138c06bdb72551b [term 1 FOLLOWER]: Refusing update from remote peer 64741c853ca84abc8a3dc8a57c5948db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:02.169584 18582 raft_consensus.cc:1261] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 1 FOLLOWER]: Refusing update from remote peer 64741c853ca84abc8a3dc8a57c5948db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:02.170316 18717 consensus_queue.cc:1035] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [LEADER]: Connected to new peer: Peer: permanent_uuid: "3544aaa9a1eb4e23a138c06bdb72551b" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:02.170522 18722 consensus_queue.cc:1035] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0d0a07f5e324ba4a7ec4e17cf9f2706" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 40055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:02.177516 18726 mvcc.cc:205] Tried to move back new op lower bound from 7317045912238575616 to 7317045912131383296. Current Snapshot: MvccSnapshot[applied={T|T < 7317045912238575616}]
I20260810 18:56:02.186017 18727 mvcc.cc:205] Tried to move back new op lower bound from 7317045912238575616 to 7317045912131383296. Current Snapshot: MvccSnapshot[applied={T|T < 7317045912238575616}]
I20260810 18:56:02.188382 18728 mvcc.cc:205] Tried to move back new op lower bound from 7317045912238575616 to 7317045912131383296. Current Snapshot: MvccSnapshot[applied={T|T < 7317045912238575616}]
I20260810 18:56:02.196897 17988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:02.197098 17988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:02.199908 17988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1d015734a0d94e9090f6842531eb3a3b: Sending DeleteTablet for 3 replicas of tablet 91512b5e38de454ea931f2a9a739a86b
I20260810 18:56:02.201086 18380 tablet_service.cc:1514] Processing DeleteTablet for tablet 91512b5e38de454ea931f2a9a739a86b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:02 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:37330
I20260810 18:56:02.201583 18576 tablet_service.cc:1514] Processing DeleteTablet for tablet 91512b5e38de454ea931f2a9a739a86b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:02 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:47550
I20260810 18:56:02.202212 18197 tablet_service.cc:1514] Processing DeleteTablet for tablet 91512b5e38de454ea931f2a9a739a86b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:02 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:60444
I20260810 18:56:02.203140 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 18046
I20260810 18:56:02.210741 18744 tablet_replica.cc:330] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: stopping tablet replica
I20260810 18:56:02.211295 18744 raft_consensus.cc:2229] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:56:02.211861 18744 raft_consensus.cc:2258] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:02.213730 18744 ts_tablet_manager.cc:1905] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:02.217624 18744 ts_tablet_manager.cc:1918] T 91512b5e38de454ea931f2a9a739a86b P 64741c853ca84abc8a3dc8a57c5948db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260810 18:56:02.221419 17964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:46157 (error 108)
I20260810 18:56:02.222345 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 18299
I20260810 18:56:02.228549 18746 tablet_replica.cc:330] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: stopping tablet replica
I20260810 18:56:02.228994 18746 raft_consensus.cc:2229] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:56:02.229936 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:02.229564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46157 (local address 127.0.0.1:60452)
0810 18:56:02.229768 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:02.229842 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:46157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260810 18:56:02.230144 17964 catalog_manager.cc:4584] TS 3544aaa9a1eb4e23a138c06bdb72551b (127.30.153.129:46157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91512b5e38de454ea931f2a9a739a86b: Network error: Client connection negotiation failed: client connection to 127.30.153.129:46157: connect: Connection refused (error 111)
I20260810 18:56:02.230753 18746 raft_consensus.cc:2258] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:02.232322 18746 ts_tablet_manager.cc:1905] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:02.234920 18746 ts_tablet_manager.cc:1918] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:02.235083 18746 log.cc:1196] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/91512b5e38de454ea931f2a9a739a86b
I20260810 18:56:02.235510 18746 ts_tablet_manager.cc:1939] T 91512b5e38de454ea931f2a9a739a86b P b0d0a07f5e324ba4a7ec4e17cf9f2706: Deleting consensus metadata
I20260810 18:56:02.236042 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 18481
I20260810 18:56:02.241087 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:02.236295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:44439 (local address 127.0.0.1:37344)
0810 18:56:02.240946 (+ 4651us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:02.241006 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.130:44439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4543}
I20260810 18:56:02.257999 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 17933
I20260810 18:56:02.269155 31334 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 18:56:02.614634 31334 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 18:56:02.614769 31334 mini_ranger.cc:66] Stopping Ranger...
I20260810 18:56:02.970628 31334 mini_ranger.cc:68] Stopped Ranger
2026-08-10 18:56:02.970 UTC [16520] LOG: received smart shutdown request
2026-08-10 18:56:02.979 UTC [16520] LOG: background worker "logical replication launcher" (PID 16533) exited with exit code 1
2026-08-10 18:56:02.980 UTC [16525] LOG: shutting down
2026-08-10 18:56:03.171 UTC [16520] LOG: database system is shut down
2026-08-10T18:56:03Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52852 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:56:03 dist-test-slave-68xk krb5kdc[18763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:56:03 dist-test-slave-68xk krb5kdc[18763](info): set up 2 sockets
Aug 10 18:56:03 dist-test-slave-68xk krb5kdc[18763](info): commencing operation
krb5kdc: starting...
W20260810 18:56:04.582131 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:56:04 dist-test-slave-68xk krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:56:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:56:04.717595 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38329
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38329,127.30.153.189:45099,127.30.153.188:41127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:04.833043 18782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:04.833364 18782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:04.833442 18782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:04.837559 18782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:04.837648 18782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:04.837688 18782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:04.837745 18782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:04.837782 18782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:04.843547 18782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38329,127.30.153.189:45099,127.30.153.188:41127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:04.844805 18782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:04.845978 18782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:04.855237 18793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:04.855233 18788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:04.855230 18789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:04.855558 18782 server_base.cc:1034] running on GCE node
I20260810 18:56:04.856326 18782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:04.857527 18782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:04.858726 18782 hybrid_clock.cc:648] HybridClock initialized: now 1786388164858702 us; error 39 us; skew 500 ppm
Aug 10 18:56:04 dist-test-slave-68xk krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:04.862036 18782 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:56:04.863099 18782 webserver.cc:460] Webserver started at http://127.30.153.190:34619/ using document root <none> and password file <none>
I20260810 18:56:04.863677 18782 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:04.863746 18782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:04.863926 18782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:04.866153 18782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "edf9a18cc0a94075adb629b6983bd6cc"
format_stamp: "Formatted at 2026-08-10 18:56:04 on dist-test-slave-68xk"
server_key: "62e1db26fc41cbee5fe883eaf83d9f82"
server_key_iv: "716c51c4e9a5efeab3e3dced478bed81"
server_key_version: "encryptionkey@0"
I20260810 18:56:04.866804 18782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "edf9a18cc0a94075adb629b6983bd6cc"
format_stamp: "Formatted at 2026-08-10 18:56:04 on dist-test-slave-68xk"
server_key: "62e1db26fc41cbee5fe883eaf83d9f82"
server_key_iv: "716c51c4e9a5efeab3e3dced478bed81"
server_key_version: "encryptionkey@0"
I20260810 18:56:04.870886 18782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:56:04.873656 18806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:04.875197 18782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:56:04.875349 18782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "edf9a18cc0a94075adb629b6983bd6cc"
format_stamp: "Formatted at 2026-08-10 18:56:04 on dist-test-slave-68xk"
server_key: "62e1db26fc41cbee5fe883eaf83d9f82"
server_key_iv: "716c51c4e9a5efeab3e3dced478bed81"
server_key_version: "encryptionkey@0"
I20260810 18:56:04.875468 18782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:04.899133 18782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:04.902745 18782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:04.902972 18782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:04.910710 18782 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:38329
I20260810 18:56:04.911044 18874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:38329 every 8 connection(s)
I20260810 18:56:04.911753 18782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:56:04.914626 18876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:04.915306 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 18782
I20260810 18:56:04.915494 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:56:04.915820 31334 external_mini_cluster.cc:1428] Setting key 48cbf10cd66be1c475c2a9c0d217b5a8
I20260810 18:56:04.918658 18876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.153.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.189@KRBTEST.COM" created.
Aug 10 18:56:04 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:04.936815 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:04.921261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54670 (local address 127.30.153.190:38329)
0810 18:56:04.921835 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:04.921853 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:04.922053 (+ 200us) server_negotiation.cc:407] Connection header received
0810 18:56:04.923545 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:04.923550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:04.923725 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:04.923877 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:04.925017 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:04.925965 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:04.927300 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:04.927670 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:04.927932 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:04.931016 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:04.931027 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:04.931041 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:04.931073 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:04.933333 (+ 2260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:04.933939 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:04.933948 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:04.933953 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:04.934104 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:04.934552 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:04.934556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:04.934559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:04.935136 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:04.935298 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:04.935746 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 18:56:04.936052 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":449,"spinlock_wait_cycles":54016,"thread_start_us":154,"threads_started":1}
I20260810 18:56:04.936813 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:04.921261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38329 (local address 127.0.0.1:54670)
0810 18:56:04.921857 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:04.921919 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:04.923211 (+ 1292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:04.923947 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:04.923955 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:04.924442 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:04.924833 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:04.924857 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:04.926179 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:04.926182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:04.927150 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:04.927161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:04.927946 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:04.927949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:04.928069 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:04.928726 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:04.928755 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:04.930799 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:04.933483 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:04.933493 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:04.933501 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:04.933792 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:04.934240 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:04.934242 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:04.934244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:04.934362 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:04.935292 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:04.935298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:04.935435 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 18:56:04.935594 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 18:56:04.935903 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":177,"threads_started":1}
I20260810 18:56:04.940181 18876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:56:04.941075 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:04.940745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51468)
0810 18:56:04.940927 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:04.941008 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260810 18:56:04.941334 18810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
I20260810 18:56:04.942380 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:04.942121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51472)
0810 18:56:04.942272 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:04.942323 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260810 18:56:04.942651 18876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:45099: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab.
Entry for principal kudu/127.30.153.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab.
I20260810 18:56:04.949491 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.189:45099
--webserver_interface=127.30.153.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab
--principal=kudu/127.30.153.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38329,127.30.153.189:45099,127.30.153.188:41127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:04.991933 18876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } attempt: 1
I20260810 18:56:04.992962 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:04.992624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51478)
0810 18:56:04.992801 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:04.992880 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 18:56:04.993836 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:04.993588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51480)
0810 18:56:04.993729 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:04.993776 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:56:04.993973 18876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:45099: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
I20260810 18:56:05.062247 18876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } attempt: 2
I20260810 18:56:05.063172 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.062861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51490)
0810 18:56:05.063018 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.063094 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 18:56:05.064062 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.063805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51504)
0810 18:56:05.063942 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.064002 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 18:56:05.064201 18876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:45099: Network error: Client connection negotiation failed: client connection to 127.30.153.189:45099: connect: Connection refused (error 111)
W20260810 18:56:05.073648 18890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:05.073963 18890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:05.074054 18890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:05.078109 18890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:05.078212 18890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:05.078253 18890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:05.078307 18890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:05.078342 18890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:05.084000 18890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38329,127.30.153.189:45099,127.30.153.188:41127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab
--principal=kudu/127.30.153.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.189:45099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.153.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:05.085158 18890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:05.086241 18890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:05.095283 18897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:05.095361 18890 server_base.cc:1034] running on GCE node
W20260810 18:56:05.095273 18898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:05.095273 18901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:05.096282 18890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:05.097465 18890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:05.098663 18890 hybrid_clock.cc:648] HybridClock initialized: now 1786388165098645 us; error 37 us; skew 500 ppm
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388165, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:05.101666 18890 init.cc:377] Logged in from keytab as kudu/127.30.153.189@KRBTEST.COM (short username kudu)
I20260810 18:56:05.102659 18890 webserver.cc:460] Webserver started at http://127.30.153.189:39859/ using document root <none> and password file <none>
I20260810 18:56:05.103255 18890 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:05.103363 18890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:05.103608 18890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:05.105726 18890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/data/instance:
uuid: "dddeb03c0353443098011d915b19a82f"
format_stamp: "Formatted at 2026-08-10 18:56:05 on dist-test-slave-68xk"
server_key: "ded6fb38a9d868a0eabd792499e8197e"
server_key_iv: "3f67b23e1590c2e0444165d17bb8603b"
server_key_version: "encryptionkey@0"
I20260810 18:56:05.106374 18890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/wal/instance:
uuid: "dddeb03c0353443098011d915b19a82f"
format_stamp: "Formatted at 2026-08-10 18:56:05 on dist-test-slave-68xk"
server_key: "ded6fb38a9d868a0eabd792499e8197e"
server_key_iv: "3f67b23e1590c2e0444165d17bb8603b"
server_key_version: "encryptionkey@0"
I20260810 18:56:05.110356 18890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 18:56:05.112843 18912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:05.114064 18890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:56:05.114233 18890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/wal
uuid: "dddeb03c0353443098011d915b19a82f"
format_stamp: "Formatted at 2026-08-10 18:56:05 on dist-test-slave-68xk"
server_key: "ded6fb38a9d868a0eabd792499e8197e"
server_key_iv: "3f67b23e1590c2e0444165d17bb8603b"
server_key_version: "encryptionkey@0"
I20260810 18:56:05.114354 18890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:05.140381 18890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:05.143601 18890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:05.143846 18890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:05.151360 18890 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.189:45099
I20260810 18:56:05.151660 18981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.189:45099 every 8 connection(s)
I20260810 18:56:05.152448 18890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
I20260810 18:56:05.155359 18876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } attempt: 3
I20260810 18:56:05.155611 18983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:05.157222 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 18890
I20260810 18:56:05.157385 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-1/wal/instance
I20260810 18:56:05.157691 31334 external_mini_cluster.cc:1428] Setting key f4fcd11283f2428ac097530eb3c23354
I20260810 18:56:05.161201 18983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.188@KRBTEST.COM; defaulting to no policy
I20260810 18:56:05.169715 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.155918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51510)
0810 18:56:05.156115 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.156136 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:05.156246 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:05.158915 (+ 2669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:05.158921 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:05.159155 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:05.159345 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.159357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.161334 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.161336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.161982 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.161995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.162820 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.162825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.162951 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.163663 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:05.163684 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:05.165243 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:05.167809 (+ 2566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.167813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.167816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.168025 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.168388 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.168391 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.168393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.168479 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.169247 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:05.169253 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:05.169381 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 18:56:05.169471 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 18:56:05.169583 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":5,"spinlock_wait_cycles":1792}
I20260810 18:56:05.171211 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.156564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.30.153.189:45099)
0810 18:56:05.157017 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.157041 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.157088 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:56:05.158032 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.158057 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.158533 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.158954 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.159520 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.161171 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.162120 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.162674 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.162873 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.165441 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.165446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.165454 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.165481 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.167693 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.168153 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.168158 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.168163 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.168277 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.168575 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.168580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.168582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.169103 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.169937 (+ 834us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.170087 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.170361 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.188@KRBTEST.COM" created.
I20260810 18:56:05.173132 18876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388165, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:05.174935 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.174640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45128)
0810 18:56:05.174785 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.174835 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 18:56:05.175976 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.175591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45130)
0810 18:56:05.175817 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.175877 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260810 18:56:05.176136 18876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:41127: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
I20260810 18:56:05.177407 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.163519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.30.153.190:38329)
0810 18:56:05.163692 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.163699 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.164187 (+ 488us) server_negotiation.cc:407] Connection header received
0810 18:56:05.164414 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.164421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.164484 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.164572 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.165691 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.167290 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.168782 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.169180 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.169313 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.173863 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.173867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.173870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.173899 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.175553 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.176022 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.176026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.176028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.176212 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.176496 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.176500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.176502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.176682 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.176795 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.177144 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.177259 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":62,"spinlock_wait_cycles":4352}
I20260810 18:56:05.177434 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.163482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38329 (local address 127.0.0.1:54686)
0810 18:56:05.164027 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.164093 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:05.164291 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:05.164668 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:05.164676 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:05.165287 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:05.165529 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.165543 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.167444 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.167450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.168638 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.168653 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.169309 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.169313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.169467 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.170290 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:05.170312 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:05.173724 (+ 3412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:05.175669 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.175678 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.175684 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.175915 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.176304 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.176307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.176309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.176398 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.176807 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:05.176815 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:05.176931 (+ 116us) client_negotiation.cc:769] Sending connection context
0810 18:56:05.177074 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 18:56:05.177271 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":224,"threads_started":1}
I20260810 18:56:05.178766 18983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388165, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
I20260810 18:56:05.189010 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.179387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51512)
0810 18:56:05.179486 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.179500 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:05.179577 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:05.179795 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:05.179798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:05.179955 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:05.180038 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.180043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.180895 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.180900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.181406 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.181413 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.181880 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.181883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.181980 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.182526 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:05.182537 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:05.185692 (+ 3155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:05.187538 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.187543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.187545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.187747 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.188023 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.188028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.188030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.188110 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.188478 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:05.188481 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:05.188573 (+ 92us) client_negotiation.cc:769] Sending connection context
0810 18:56:05.188679 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 18:56:05.188825 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260810 18:56:05.189010 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.179471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51512 (local address 127.30.153.189:45099)
0810 18:56:05.179583 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.179587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.179600 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:56:05.179656 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.179659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.179693 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.179785 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.180146 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.180790 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.181520 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.181765 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.181868 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.185850 (+ 3982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.185853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.185855 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.185875 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.187419 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.187846 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.187852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.187854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.187930 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.188202 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.188207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.188208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.188384 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.188490 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.188734 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.188884 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":52}
I20260810 18:56:05.189970 18983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:56:05.190721 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.190407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45140)
0810 18:56:05.190575 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.190658 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:56:05.191530 18916 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab.
Entry for principal kudu/127.30.153.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab.
I20260810 18:56:05.192868 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.192569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45152)
0810 18:56:05.192736 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.192791 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260810 18:56:05.193018 18983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:41127: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
I20260810 18:56:05.193928 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.188:41127
--webserver_interface=127.30.153.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab
--principal=kudu/127.30.153.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38329,127.30.153.189:45099,127.30.153.188:41127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:05.207324 18876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } attempt: 1
I20260810 18:56:05.208290 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.207965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45164)
0810 18:56:05.208139 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.208206 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 18:56:05.209120 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.208860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45170)
0810 18:56:05.208991 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.209050 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 18:56:05.209275 18876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:41127: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
I20260810 18:56:05.242377 18983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } attempt: 1
I20260810 18:56:05.243345 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.243017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45186)
0810 18:56:05.243179 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.243259 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 18:56:05.244318 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.244035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45200)
0810 18:56:05.244191 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.244249 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260810 18:56:05.244534 18983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:41127: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
I20260810 18:56:05.284461 18876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } attempt: 2
I20260810 18:56:05.285441 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.285064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45216)
0810 18:56:05.285277 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.285351 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260810 18:56:05.286374 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.286080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45230)
0810 18:56:05.286244 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.286308 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260810 18:56:05.286556 18876 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:41127: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
I20260810 18:56:05.312753 18983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } attempt: 2
I20260810 18:56:05.313802 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.313451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45244)
0810 18:56:05.313643 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.313714 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 18:56:05.314769 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.314514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45246)
0810 18:56:05.314634 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.314704 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:56:05.314934 18983 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:41127: Network error: Client connection negotiation failed: client connection to 127.30.153.188:41127: connect: Connection refused (error 111)
W20260810 18:56:05.314981 18995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:05.315300 18995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:05.315393 18995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:05.319494 18995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:05.319571 18995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:05.319617 18995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:05.319656 18995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:05.319691 18995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:05.324936 18995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38329,127.30.153.189:45099,127.30.153.188:41127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab
--principal=kudu/127.30.153.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.188:41127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.153.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:05.326041 18995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:05.327159 18995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:05.336797 19001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:05.336742 19003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:05.337091 19000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:05.337160 18995 server_base.cc:1034] running on GCE node
I20260810 18:56:05.337783 18995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:05.339015 18995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:05.340209 18995 hybrid_clock.cc:648] HybridClock initialized: now 1786388165340192 us; error 34 us; skew 500 ppm
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388165, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:05.343072 18995 init.cc:377] Logged in from keytab as kudu/127.30.153.188@KRBTEST.COM (short username kudu)
I20260810 18:56:05.344043 18995 webserver.cc:460] Webserver started at http://127.30.153.188:45423/ using document root <none> and password file <none>
I20260810 18:56:05.344568 18995 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:05.344627 18995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:05.344797 18995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:05.346873 18995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/data/instance:
uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab"
format_stamp: "Formatted at 2026-08-10 18:56:05 on dist-test-slave-68xk"
server_key: "5d25a72ed49b08e078e23f5c199f710a"
server_key_iv: "ad35882559563e85c309349ce2b81757"
server_key_version: "encryptionkey@0"
I20260810 18:56:05.347566 18995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/wal/instance:
uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab"
format_stamp: "Formatted at 2026-08-10 18:56:05 on dist-test-slave-68xk"
server_key: "5d25a72ed49b08e078e23f5c199f710a"
server_key_iv: "ad35882559563e85c309349ce2b81757"
server_key_version: "encryptionkey@0"
I20260810 18:56:05.351807 18995 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:56:05.354612 19011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:05.355993 18995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:56:05.356204 18995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/wal
uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab"
format_stamp: "Formatted at 2026-08-10 18:56:05 on dist-test-slave-68xk"
server_key: "5d25a72ed49b08e078e23f5c199f710a"
server_key_iv: "ad35882559563e85c309349ce2b81757"
server_key_version: "encryptionkey@0"
I20260810 18:56:05.356338 18995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:05.370347 18995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:05.373740 18995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:05.373947 18995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:05.381176 18995 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.188:41127
I20260810 18:56:05.381496 19089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.188:41127 every 8 connection(s)
I20260810 18:56:05.382181 18995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
I20260810 18:56:05.384953 19090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:05.385730 18876 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } attempt: 3
I20260810 18:56:05.390237 19090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:56:05.391901 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 18995
I20260810 18:56:05.392042 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786387990686100-31334-0/minicluster-data/master-2/wal/instance
I20260810 18:56:05.392344 31334 external_mini_cluster.cc:1428] Setting key 770f8d04feb122ca52c8157633b55b20
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388165, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:05.399151 18881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.386280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45258)
0810 18:56:05.386474 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.386494 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:05.386604 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:05.388787 (+ 2183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:05.388791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:05.388999 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:05.389107 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.389115 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.390448 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.390452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.391040 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.391051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.391832 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.391836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.391958 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.392616 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:05.392635 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:05.394516 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:05.396949 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.396953 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.396955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.397218 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.397646 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.397650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.397651 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.397764 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.398526 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:05.398529 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:05.398641 (+ 112us) client_negotiation.cc:769] Sending connection context
0810 18:56:05.398760 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 18:56:05.398916 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
I20260810 18:56:05.400252 19092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.387005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.30.153.188:41127)
0810 18:56:05.387379 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.387394 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.387429 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:56:05.388146 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.388163 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.388512 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.388845 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.389289 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.390275 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.391166 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.391673 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.392282 (+ 609us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.394686 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.394692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.394704 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.394733 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.396828 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.397359 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.397367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.397375 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.397506 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.397892 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.397896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.397898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.398391 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.398553 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.398951 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.399232 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
I20260810 18:56:05.403757 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.392208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.30.153.190:38329)
0810 18:56:05.392531 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.392536 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.393037 (+ 501us) server_negotiation.cc:407] Connection header received
0810 18:56:05.393260 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.393263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.393315 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.393366 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.394434 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.395168 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.396567 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.396907 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.397017 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.399570 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.399573 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.399576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.399600 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.401836 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.402337 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.402341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.402342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.402464 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.402792 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.402797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.402799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.403016 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.403134 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.403467 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.403588 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234}
I20260810 18:56:05.403774 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.391968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38329 (local address 127.0.0.1:54702)
0810 18:56:05.392910 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.392963 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:05.393132 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:05.393571 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:05.393576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:05.394080 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:05.394273 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.394281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.395339 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.395345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.396434 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.396445 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.397023 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.397026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.397154 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.397687 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:05.397705 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:05.399408 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:05.401961 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.401968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.401972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.402247 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.402578 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.402581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.402583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.402687 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.403134 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:05.403142 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:05.403263 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 18:56:05.403408 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 18:56:05.403590 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":43,"thread_start_us":157,"threads_started":1}
I20260810 18:56:05.405099 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.393340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45270 (local address 127.30.153.188:41127)
0810 18:56:05.393663 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.393674 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.394113 (+ 439us) server_negotiation.cc:407] Connection header received
0810 18:56:05.394279 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.394282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.394329 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.394438 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.395445 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.396178 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.397323 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.397706 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.397830 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.400783 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.400787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.400789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.400821 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.403030 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.403513 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.403516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.403518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.403599 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.404226 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.404229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.404230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.404394 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.404528 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.404791 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.404894 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":235,"thread_start_us":98,"threads_started":1}
I20260810 18:56:05.405313 18876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc: Bootstrap starting.
I20260810 18:56:05.406127 18983 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } attempt: 3
I20260810 18:56:05.407847 19090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:56:05.409314 18876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:05.410377 18876 log.cc:824] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388165, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
I20260810 18:56:05.412633 18876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc: No bootstrap required, opened a new log
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388165, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
I20260810 18:56:05.417585 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.406851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.30.153.190:38329)
0810 18:56:05.406959 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.406965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.406977 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:56:05.407010 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.407012 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.407052 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.407124 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.407964 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.408597 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.409670 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.409986 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.410064 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.413810 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.413814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.413817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.413842 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.415543 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.416051 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.416055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.416056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.416141 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.416445 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.416448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.416450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.416691 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.416855 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.417276 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.417421 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260810 18:56:05.417757 18876 raft_consensus.cc:348] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.418188 18876 raft_consensus.cc:374] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:05.418229 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.408365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51524 (local address 127.30.153.189:45099)
0810 18:56:05.408476 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.408480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.408571 (+ 91us) server_negotiation.cc:407] Connection header received
0810 18:56:05.408736 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.408739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.408782 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.408830 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.409370 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.410067 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.411238 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.411588 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.411674 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.414001 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.414005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.414007 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.414029 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.415851 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.416468 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.416474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.416476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.416567 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.416927 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.416933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.416936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.417316 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.417411 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.417915 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.418041 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:56:05.418468 18876 raft_consensus.cc:729] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf9a18cc0a94075adb629b6983bd6cc, State: Initialized, Role: FOLLOWER
I20260810 18:56:05.418752 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.408295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:45099 (local address 127.0.0.1:51524)
0810 18:56:05.408500 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.408513 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:05.408638 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:05.408950 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:05.408952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:05.409126 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:05.409239 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.409245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.410189 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.410191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.411075 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.411088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.411826 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.411831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.411953 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.412431 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:05.412446 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:05.413843 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:05.415992 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.415996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.415999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.416333 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.416699 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.416702 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.416704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.416802 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.417502 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:05.417507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:05.417628 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 18:56:05.418398 (+ 770us) client_negotiation.cc:240] Negotiation successful
0810 18:56:05.418560 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260810 18:56:05.419530 19090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:56:05.419746 18876 consensus_queue.cc:260] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.421077 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.406659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45272)
0810 18:56:05.406791 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.406807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:05.406892 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:05.407212 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:05.407215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:05.407416 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:05.407528 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.407535 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.408650 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.408655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.409469 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.409478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.410081 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.410086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.410182 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.410946 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:05.410961 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:05.413900 (+ 2939us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:05.418676 (+ 4776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.418679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.418681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.418955 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.419301 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.419304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.419306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.419404 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.420608 (+ 1204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:05.420611 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:05.420704 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 18:56:05.420801 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:56:05.420933 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 18:56:05.421583 19096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.406695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.30.153.188:41127)
0810 18:56:05.406822 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.406827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.406842 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:56:05.407009 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.407012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.407059 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.407645 (+ 586us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.407745 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.408531 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.409601 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.409959 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.414026 (+ 4067us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.415138 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.415143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.415146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.415175 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.418166 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.419074 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.419078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.419079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.419187 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.419525 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.419528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.419529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.419765 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.421260 (+ 1495us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.421336 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.421448 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:56:05.421707 18876 sys_catalog.cc:564] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:05.425345 18983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f: Bootstrap starting.
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388165, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
Aug 10 18:56:05 dist-test-slave-68xk krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
I20260810 18:56:05.427680 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.428731 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:56:05.428762 18983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:05.429699 18983 log.cc:824] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:05.430768 19092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.420215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45280 (local address 127.30.153.188:41127)
0810 18:56:05.420338 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.420343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.420354 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:56:05.420386 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.420389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.420424 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.420504 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.420986 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.421770 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.423022 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.423374 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.423441 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.426657 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.426662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.426664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.426693 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.428368 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.428835 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.428838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.428840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.428921 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.429211 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.429214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.429215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.429466 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.429544 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.430470 (+ 926us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.430604 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:56:05.431283 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.419865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.30.153.189:45099)
0810 18:56:05.419970 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.419974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.420066 (+ 92us) server_negotiation.cc:407] Connection header received
0810 18:56:05.420256 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.420260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.420305 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.420358 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.421247 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.421946 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.423036 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.423360 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.423440 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.426692 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.426696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.426698 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.426724 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.428590 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.429762 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.429766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.429769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.429867 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.430199 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.430202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.430204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.430452 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.430614 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.431028 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.431142 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 18:56:05.431293 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.420013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:41127 (local address 127.0.0.1:45280)
0810 18:56:05.420115 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:05.420128 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:05.420196 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:05.420550 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:05.420555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:05.420732 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:05.420850 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.420858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.422034 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.422036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.422887 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:05.422898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.423579 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.423581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:05.423655 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.424270 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:05.424291 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:05.426483 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:05.428484 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.428486 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.428488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.428729 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.429022 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:05.429024 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:05.429027 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:05.429120 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:05.429610 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:05.429613 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:05.429710 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 18:56:05.431011 (+ 1301us) client_negotiation.cc:240] Negotiation successful
0810 18:56:05.431142 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260810 18:56:05.432392 18983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f: No bootstrap required, opened a new log
W20260810 18:56:05.434952 19112 catalog_manager.cc:1558] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:56:05.435063 19112 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:56:05.435599 19090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab: Bootstrap starting.
I20260810 18:56:05.436645 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:56:05.436530 18983 raft_consensus.cc:348] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.436743 18983 raft_consensus.cc:374] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:05.436805 18983 raft_consensus.cc:729] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dddeb03c0353443098011d915b19a82f, State: Initialized, Role: FOLLOWER
I20260810 18:56:05.437294 18983 consensus_queue.cc:260] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.438724 18983 sys_catalog.cc:564] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:05.430667 18782 master_runner.cc:186] Error getting master registration for 127.30.153.189:45099: OK, instance_id { permanent_uuid: "dddeb03c0353443098011d915b19a82f" instance_seqno: 1786388165149233 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:56:05.441807 18782 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:56:05.442139 19090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:05.443363 19090 log.cc:824] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab: Log is configured to *not* fsync() on all Append() calls
W20260810 18:56:05.444554 19140 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:56:05.444710 19140 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:56:05.446256 19090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab: No bootstrap required, opened a new log
I20260810 18:56:05.450773 19090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.451123 19090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:05.451238 19090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55e36bfc0c364ef6b469c7bc6fd8aaab, State: Initialized, Role: FOLLOWER
I20260810 18:56:05.447559 18890 master_runner.cc:186] Error getting master registration for 127.30.153.188:41127: OK, instance_id { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" instance_seqno: 1786388165379100 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:56:05.451812 18890 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:56:05.451828 19090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.453321 19090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:05.456351 19147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.456565 19147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:56:05.456960 19119 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.457084 19092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.445319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.30.153.188:41127)
0810 18:56:05.445437 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.445443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.447347 (+ 1904us) server_negotiation.cc:407] Connection header received
0810 18:56:05.447545 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.447548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.447592 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.447642 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.448673 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.449448 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.450667 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.450969 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.451051 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.453126 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.453130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.453132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.453158 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.454953 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.455520 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.455524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.455525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.455627 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.455979 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.455983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.455985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.456254 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.456392 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.456791 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.456924 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:56:05.457155 19119 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [sys.catalog]: This master's current role is: FOLLOWER
W20260810 18:56:05.460848 19163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:56:05.460964 19163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:56:05.463167 18995 master_runner.cc:263] No leader master found from master 55e36bfc0c364ef6b469c7bc6fd8aaab
I20260810 18:56:05.463230 18995 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:56:05.466681 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.445889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.30.153.189:45099)
0810 18:56:05.446008 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.446013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.457908 (+ 11895us) server_negotiation.cc:407] Connection header received
0810 18:56:05.458074 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.458077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.458123 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.458174 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.459081 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.459779 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.460817 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.461146 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.461230 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.463015 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.463019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.463021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.463050 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.464717 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.465233 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.465245 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.465247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.465337 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.465662 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.465666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.465668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.465880 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.465993 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.466355 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.466521 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:56:05.472769 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:05.446334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.30.153.190:38329)
0810 18:56:05.446483 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:05.446488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:05.454720 (+ 8232us) server_negotiation.cc:407] Connection header received
0810 18:56:05.456822 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:05.456826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:05.456880 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:05.456936 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:05.457860 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.458618 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.466515 (+ 7897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:05.466871 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:05.466965 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:05.468781 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:05.468785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:05.468788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:05.468817 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:05.470601 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.471094 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.471098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.471099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.471200 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:05.471497 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:05.471500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:05.471502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:05.471717 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:05.471840 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:05.472472 (+ 632us) server_negotiation.cc:299] Negotiation successful
0810 18:56:05.472607 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:56:05.540614 19100 raft_consensus.cc:482] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:05.540861 19100 raft_consensus.cc:504] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.541966 19100 leader_election.cc:288] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dddeb03c0353443098011d915b19a82f (127.30.153.189:45099), 55e36bfc0c364ef6b469c7bc6fd8aaab (127.30.153.188:41127)
I20260810 18:56:05.542908 19056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "edf9a18cc0a94075adb629b6983bd6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" is_pre_election: true
I20260810 18:56:05.543268 19056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edf9a18cc0a94075adb629b6983bd6cc in term 0.
I20260810 18:56:05.543843 18811 leader_election.cc:302] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e36bfc0c364ef6b469c7bc6fd8aaab, edf9a18cc0a94075adb629b6983bd6cc; no voters:
I20260810 18:56:05.544246 19100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:56:05.544335 19100 raft_consensus.cc:482] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:56:05.544378 19100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:05.545774 19100 raft_consensus.cc:504] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.546099 19100 leader_election.cc:288] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [CANDIDATE]: Term 1 election: Requested vote from peers dddeb03c0353443098011d915b19a82f (127.30.153.189:45099), 55e36bfc0c364ef6b469c7bc6fd8aaab (127.30.153.188:41127)
I20260810 18:56:05.546619 19056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "edf9a18cc0a94075adb629b6983bd6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab"
I20260810 18:56:05.546734 19056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:05.548022 19056 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edf9a18cc0a94075adb629b6983bd6cc in term 1.
I20260810 18:56:05.548400 18811 leader_election.cc:302] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55e36bfc0c364ef6b469c7bc6fd8aaab, edf9a18cc0a94075adb629b6983bd6cc; no voters:
I20260810 18:56:05.549121 19100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:05.549144 18949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "edf9a18cc0a94075adb629b6983bd6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dddeb03c0353443098011d915b19a82f" is_pre_election: true
I20260810 18:56:05.549466 18949 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edf9a18cc0a94075adb629b6983bd6cc in term 0.
I20260810 18:56:05.549154 18948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "edf9a18cc0a94075adb629b6983bd6cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dddeb03c0353443098011d915b19a82f"
I20260810 18:56:05.549782 18948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:05.551113 18948 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edf9a18cc0a94075adb629b6983bd6cc in term 1.
I20260810 18:56:05.551401 19100 raft_consensus.cc:686] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [term 1 LEADER]: Becoming Leader. State: Replica: edf9a18cc0a94075adb629b6983bd6cc, State: Running, Role: LEADER
I20260810 18:56:05.552642 19100 consensus_queue.cc:237] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } }
I20260810 18:56:05.555439 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader edf9a18cc0a94075adb629b6983bd6cc. Latest consensus state: current_term: 1 leader_uuid: "edf9a18cc0a94075adb629b6983bd6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.555608 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:05.557368 19170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:56:05.558311 19170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:56:05.563988 19056 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [term 1 FOLLOWER]: Refusing update from remote peer edf9a18cc0a94075adb629b6983bd6cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:05.564746 19166 consensus_queue.cc:1035] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:05.568935 19147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [sys.catalog]: SysCatalogTable state changed. Reason: New leader edf9a18cc0a94075adb629b6983bd6cc. Latest consensus state: current_term: 1 leader_uuid: "edf9a18cc0a94075adb629b6983bd6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.569068 19147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:56:05.572934 18948 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [term 1 FOLLOWER]: Refusing update from remote peer edf9a18cc0a94075adb629b6983bd6cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:05.573588 19166 consensus_queue.cc:1035] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:05.576894 19166 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "edf9a18cc0a94075adb629b6983bd6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.577029 19166 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:05.577888 19119 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [sys.catalog]: SysCatalogTable state changed. Reason: New leader edf9a18cc0a94075adb629b6983bd6cc. Latest consensus state: current_term: 1 leader_uuid: "edf9a18cc0a94075adb629b6983bd6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.578020 19119 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:56:05.578490 19147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "edf9a18cc0a94075adb629b6983bd6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.578629 19147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:56:05.579093 19171 mvcc.cc:205] Tried to move back new op lower bound from 7317045926142574592 to 7317045926107033600. Current Snapshot: MvccSnapshot[applied={T|T < 7317045926142574592}]
I20260810 18:56:05.579672 19173 mvcc.cc:205] Tried to move back new op lower bound from 7317045926142574592 to 7317045926107033600. Current Snapshot: MvccSnapshot[applied={T|T < 7317045926142574592}]
I20260810 18:56:05.586077 19170 catalog_manager.cc:1347] Generated new cluster ID: 5a00d40d4ecc4a2799305e98c0046577
I20260810 18:56:05.586144 19170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:56:05.607609 19100 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "edf9a18cc0a94075adb629b6983bd6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.607775 19100 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:05.609146 19119 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "edf9a18cc0a94075adb629b6983bd6cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edf9a18cc0a94075adb629b6983bd6cc" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38329 } } peers { permanent_uuid: "dddeb03c0353443098011d915b19a82f" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 45099 } } peers { permanent_uuid: "55e36bfc0c364ef6b469c7bc6fd8aaab" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 41127 } } }
I20260810 18:56:05.609292 19119 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:56:05.611466 19175 mvcc.cc:205] Tried to move back new op lower bound from 7317045926142574592 to 7317045926107033600. Current Snapshot: MvccSnapshot[applied={T|T < 7317045926142574592}]
I20260810 18:56:05.622807 19170 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:56:05.624185 19170 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:56:05.644486 19170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edf9a18cc0a94075adb629b6983bd6cc: Generated new TSK 0
I20260810 18:56:05.645502 19170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:56:06.446516 19140 catalog_manager.cc:1259] Loaded cluster ID: 5a00d40d4ecc4a2799305e98c0046577
I20260810 18:56:06.446619 19140 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f: loading cluster ID for follower catalog manager: success
I20260810 18:56:06.448328 19140 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f: acquiring CA information for follower catalog manager: success
I20260810 18:56:06.449503 19140 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dddeb03c0353443098011d915b19a82f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:56:06.462210 19163 catalog_manager.cc:1259] Loaded cluster ID: 5a00d40d4ecc4a2799305e98c0046577
I20260810 18:56:06.462284 19163 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab: loading cluster ID for follower catalog manager: success
I20260810 18:56:06.463968 19163 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab: acquiring CA information for follower catalog manager: success
I20260810 18:56:06.465147 19163 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 55e36bfc0c364ef6b469c7bc6fd8aaab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:56:06.712350 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 18782
I20260810 18:56:06.726622 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 18890
I20260810 18:56:06.739367 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 18995
2026-08-10T18:56:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3439 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:56:06 dist-test-slave-68xk krb5kdc[19192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:56:06 dist-test-slave-68xk krb5kdc[19192](info): set up 2 sockets
Aug 10 18:56:06 dist-test-slave-68xk krb5kdc[19192](info): commencing operation
krb5kdc: starting...
W20260810 18:56:08.019454 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:56:08 dist-test-slave-68xk krb5kdc[19192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:56:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:56:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:56:08.158227 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:36357
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:42563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:08.272702 19211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:08.272992 19211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:08.273080 19211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:08.277725 19211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:08.277804 19211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:08.277827 19211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:08.277848 19211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:08.277864 19211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:08.283627 19211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:42563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:36357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:08.284787 19211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:08.285823 19211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:08.294726 19218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:08.294720 19217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:08.294833 19211 server_base.cc:1034] running on GCE node
W20260810 18:56:08.294726 19221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:08.295718 19211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:08.296844 19211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:08.298062 19211 hybrid_clock.cc:648] HybridClock initialized: now 1786388168298035 us; error 42 us; skew 500 ppm
Aug 10 18:56:08 dist-test-slave-68xk krb5kdc[19192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388168, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:08.308645 19211 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:56:08.309628 19211 webserver.cc:460] Webserver started at http://127.30.153.190:37767/ using document root <none> and password file <none>
I20260810 18:56:08.310233 19211 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:08.310298 19211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:08.310565 19211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:08.312683 19211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "d536c9c78532418ca6d292bf6e1078a0"
format_stamp: "Formatted at 2026-08-10 18:56:08 on dist-test-slave-68xk"
server_key: "af8033d5792d8524fee11a72c446905c"
server_key_iv: "a41fb74d30eb6f124f87ef0eb4383a28"
server_key_version: "encryptionkey@0"
I20260810 18:56:08.313268 19211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "d536c9c78532418ca6d292bf6e1078a0"
format_stamp: "Formatted at 2026-08-10 18:56:08 on dist-test-slave-68xk"
server_key: "af8033d5792d8524fee11a72c446905c"
server_key_iv: "a41fb74d30eb6f124f87ef0eb4383a28"
server_key_version: "encryptionkey@0"
I20260810 18:56:08.317179 19211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 18:56:08.319720 19230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:08.320955 19211 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:56:08.321062 19211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "d536c9c78532418ca6d292bf6e1078a0"
format_stamp: "Formatted at 2026-08-10 18:56:08 on dist-test-slave-68xk"
server_key: "af8033d5792d8524fee11a72c446905c"
server_key_iv: "a41fb74d30eb6f124f87ef0eb4383a28"
server_key_version: "encryptionkey@0"
I20260810 18:56:08.321195 19211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:08.339653 19211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:08.342846 19211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:08.343176 19211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:08.351711 19211 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:36357
I20260810 18:56:08.351735 19311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:36357 every 8 connection(s)
I20260810 18:56:08.352703 19211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:56:08.355723 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 19211
I20260810 18:56:08.355871 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:56:08.356024 19312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:08.356267 31334 external_mini_cluster.cc:1428] Setting key 85aa19ff5307af0ed4cb3058ee6cba76
I20260810 18:56:08.364178 19312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0: Bootstrap starting.
Aug 10 18:56:08 dist-test-slave-68xk krb5kdc[19192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:08.366886 19312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:08.367857 19312 log.cc:824] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:08.370405 19312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0: No bootstrap required, opened a new log
I20260810 18:56:08.373476 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:08.357648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52382 (local address 127.30.153.190:36357)
0810 18:56:08.358093 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:08.358113 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:08.358160 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:56:08.359242 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:08.359268 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:08.359717 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:08.360127 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:08.360929 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:08.362585 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:08.363499 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:08.364263 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:08.364483 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:08.367470 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:08.367477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:08.367494 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:08.367527 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:08.369935 (+ 2408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:08.370404 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:08.370410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:08.370441 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:08.370596 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:08.370898 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:08.370901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:08.370903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:08.371452 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:08.371649 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:08.372010 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 18:56:08.372339 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":124,"threads_started":1}
I20260810 18:56:08.375393 19312 raft_consensus.cc:348] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d536c9c78532418ca6d292bf6e1078a0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36357 } }
I20260810 18:56:08.375690 19312 raft_consensus.cc:374] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:08.375787 19312 raft_consensus.cc:729] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d536c9c78532418ca6d292bf6e1078a0, State: Initialized, Role: FOLLOWER
I20260810 18:56:08.376286 19312 consensus_queue.cc:260] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d536c9c78532418ca6d292bf6e1078a0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36357 } }
I20260810 18:56:08.376456 19312 raft_consensus.cc:388] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:56:08.376535 19312 raft_consensus.cc:482] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:56:08.376631 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:56:08.376657 19312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:08.377934 19312 raft_consensus.cc:504] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d536c9c78532418ca6d292bf6e1078a0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36357 } }
I20260810 18:56:08.378310 19312 leader_election.cc:302] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d536c9c78532418ca6d292bf6e1078a0; no voters:
I20260810 18:56:08.378750 19312 leader_election.cc:288] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:56:08.379155 19321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:08.379412 19321 raft_consensus.cc:686] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [term 1 LEADER]: Becoming Leader. State: Replica: d536c9c78532418ca6d292bf6e1078a0, State: Running, Role: LEADER
I20260810 18:56:08.379738 19321 consensus_queue.cc:237] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d536c9c78532418ca6d292bf6e1078a0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36357 } }
I20260810 18:56:08.380108 19312 sys_catalog.cc:564] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:08.380846 19321 sys_catalog.cc:455] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d536c9c78532418ca6d292bf6e1078a0. Latest consensus state: current_term: 1 leader_uuid: "d536c9c78532418ca6d292bf6e1078a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d536c9c78532418ca6d292bf6e1078a0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36357 } } }
I20260810 18:56:08.380975 19321 sys_catalog.cc:458] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:08.381412 19329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:56:08.381412 19322 sys_catalog.cc:455] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d536c9c78532418ca6d292bf6e1078a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d536c9c78532418ca6d292bf6e1078a0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36357 } } }
I20260810 18:56:08.381628 19322 sys_catalog.cc:458] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:08.384847 19329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:56:08.390744 19329 catalog_manager.cc:1347] Generated new cluster ID: dc947b0f09314a8386dd7ce768c8f56e
I20260810 18:56:08.390830 19329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:56:08.399884 19329 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:56:08.400802 19329 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:56:08.406883 19329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d536c9c78532418ca6d292bf6e1078a0: Generated new TSK 0
I20260810 18:56:08.407567 19329 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:56:08.433516 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 19211
2026-08-10T18:56:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1697 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:56:08 dist-test-slave-68xk krb5kdc[19346](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:56:08 dist-test-slave-68xk krb5kdc[19346](info): set up 2 sockets
Aug 10 18:56:08 dist-test-slave-68xk krb5kdc[19346](info): commencing operation
krb5kdc: starting...
W20260810 18:56:09.717710 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:56:09 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:56:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:56:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:56:09.849949 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38663
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:09.964596 19365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:09.964898 19365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:09.964998 19365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:09.969049 19365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:09.969134 19365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:09.969173 19365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:09.969233 19365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:09.969277 19365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:09.974485 19365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:09.975549 19365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:09.976596 19365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:09.985677 19376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:09.985726 19365 server_base.cc:1034] running on GCE node
W20260810 18:56:09.985702 19379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:09.985677 19377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:09.986727 19365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:09.987772 19365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:09.988936 19365 hybrid_clock.cc:648] HybridClock initialized: now 1786388169988908 us; error 46 us; skew 500 ppm
Aug 10 18:56:09 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388169, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:09.991775 19365 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:56:09.992668 19365 webserver.cc:460] Webserver started at http://127.30.153.190:39095/ using document root <none> and password file <none>
I20260810 18:56:09.993234 19365 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:09.993330 19365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:09.993570 19365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:09.995728 19365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "54f600d408ad4aa2a38ced4bb38749b0"
format_stamp: "Formatted at 2026-08-10 18:56:09 on dist-test-slave-68xk"
server_key: "dd94ad69273798035910d33955c9db5c"
server_key_iv: "6caee141edbf2a8a2f7263021d9c6c84"
server_key_version: "encryptionkey@0"
I20260810 18:56:09.996299 19365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "54f600d408ad4aa2a38ced4bb38749b0"
format_stamp: "Formatted at 2026-08-10 18:56:09 on dist-test-slave-68xk"
server_key: "dd94ad69273798035910d33955c9db5c"
server_key_iv: "6caee141edbf2a8a2f7263021d9c6c84"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.000088 19365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:56:10.002933 19392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.004244 19365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:56:10.004393 19365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "54f600d408ad4aa2a38ced4bb38749b0"
format_stamp: "Formatted at 2026-08-10 18:56:09 on dist-test-slave-68xk"
server_key: "dd94ad69273798035910d33955c9db5c"
server_key_iv: "6caee141edbf2a8a2f7263021d9c6c84"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.004514 19365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:10.022451 19365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:10.025341 19365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:10.025556 19365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:10.032804 19365 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:38663
I20260810 18:56:10.033095 19476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:38663 every 8 connection(s)
I20260810 18:56:10.033896 19365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:56:10.037014 19477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:10.037101 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 19365
I20260810 18:56:10.037271 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:56:10.037604 31334 external_mini_cluster.cc:1428] Setting key f7be87430d1db229733af9137fe3f176
I20260810 18:56:10.045145 19477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Bootstrap starting.
Aug 10 18:56:10 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:10.048262 19477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:10.049052 19477 log.cc:824] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:10.051222 19477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: No bootstrap required, opened a new log
I20260810 18:56:10.052826 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.039050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.30.153.190:38663)
0810 18:56:10.039485 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:10.039499 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:10.039535 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:56:10.040152 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:10.040168 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:10.040478 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:10.040756 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:10.041793 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.042884 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.044208 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.044654 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.044775 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.047916 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:10.047921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:10.047934 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:10.047961 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:10.049803 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.050263 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.050275 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.050279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.050398 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.050734 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.050744 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.050746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.051123 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:10.051350 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:10.051649 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 18:56:10.051940 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":142,"threads_started":1}
I20260810 18:56:10.054705 19477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:10.054997 19477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:10.055073 19477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f600d408ad4aa2a38ced4bb38749b0, State: Initialized, Role: FOLLOWER
I20260810 18:56:10.055482 19477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:10.055612 19477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:56:10.055667 19477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:56:10.055742 19477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:10.056648 19477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:10.056950 19477 leader_election.cc:302] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54f600d408ad4aa2a38ced4bb38749b0; no voters:
I20260810 18:56:10.057220 19477 leader_election.cc:288] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:56:10.057355 19484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:10.057591 19484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 1 LEADER]: Becoming Leader. State: Replica: 54f600d408ad4aa2a38ced4bb38749b0, State: Running, Role: LEADER
I20260810 18:56:10.057898 19484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:10.058621 19477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:10.059793 19485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54f600d408ad4aa2a38ced4bb38749b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } } }
I20260810 18:56:10.059934 19485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:10.059796 19487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54f600d408ad4aa2a38ced4bb38749b0. Latest consensus state: current_term: 1 leader_uuid: "54f600d408ad4aa2a38ced4bb38749b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } } }
I20260810 18:56:10.060214 19487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:10.060519 19494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:56:10.063618 19494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:56:10.069831 19494 catalog_manager.cc:1347] Generated new cluster ID: 6e07aac6588d4b65a8a5b22475737136
I20260810 18:56:10.069902 19494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:56:10.079138 19494 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:56:10.080327 19494 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:56:10.090349 19494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Generated new TSK 0
I20260810 18:56:10.091142 19494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:56:10.129776 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38663
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:10.243808 19516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:10.244081 19516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:10.244127 19516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:10.248220 19516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:10.248286 19516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:10.248344 19516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:56:10.253501 19516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:10.254558 19516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:10.255584 19516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:56:10.265825 19516 server_base.cc:1034] running on GCE node
W20260810 18:56:10.265796 19522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:10.265869 19527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:10.265810 19523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:10.266474 19516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:10.267158 19516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:10.268373 19516 hybrid_clock.cc:648] HybridClock initialized: now 1786388170268341 us; error 48 us; skew 500 ppm
Aug 10 18:56:10 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388170, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:10.271374 19516 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:56:10.272269 19516 webserver.cc:460] Webserver started at http://127.30.153.129:37481/ using document root <none> and password file <none>
I20260810 18:56:10.272877 19516 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:10.272977 19516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:10.273218 19516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:10.275367 19516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "d94f3a58760c4c99b09c2cce7d283f83"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "d16afb6d674cd99eba6bffa668cc3e6f"
server_key_iv: "2414767960c999886bdf016e405216d5"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.275952 19516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "d94f3a58760c4c99b09c2cce7d283f83"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "d16afb6d674cd99eba6bffa668cc3e6f"
server_key_iv: "2414767960c999886bdf016e405216d5"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.281243 19516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 18:56:10.286361 19534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.288276 19516 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 18:56:10.288439 19516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "d94f3a58760c4c99b09c2cce7d283f83"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "d16afb6d674cd99eba6bffa668cc3e6f"
server_key_iv: "2414767960c999886bdf016e405216d5"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.288573 19516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:10.321610 19516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:10.325008 19516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:10.325255 19516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:10.325955 19516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:10.326946 19516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:10.327000 19516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.327086 19516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:10.327121 19516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.336849 19516 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:35839
I20260810 18:56:10.336879 19712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:35839 every 8 connection(s)
I20260810 18:56:10.337890 19516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:56:10.347420 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 19516
I20260810 18:56:10.347596 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
Aug 10 18:56:10 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388170, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:10.347986 31334 external_mini_cluster.cc:1428] Setting key fb40d1474d66f3b49041d58c42e61445
I20260810 18:56:10.352720 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.339775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:44305 (local address 127.30.153.190:38663)
0810 18:56:10.339917 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:10.339921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:10.340813 (+ 892us) server_negotiation.cc:407] Connection header received
0810 18:56:10.342194 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:10.342199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:10.342278 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:10.342384 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:10.343599 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.344141 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.345301 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.345545 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.345659 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.348643 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:10.348649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:10.348652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:10.348685 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:10.350567 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.351066 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.351073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.351078 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.351195 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.351579 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.351582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.351584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.351783 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:10.351899 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:10.352424 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 18:56:10.352554 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:56:10.353565 19717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.340168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:44305)
0810 18:56:10.340640 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:10.340682 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:10.341965 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:10.342597 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:10.342604 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:10.342965 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:10.343439 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.343452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.344271 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.344276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.345162 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.345171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.345701 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.345705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.345877 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.346491 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:10.346507 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:10.348409 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:10.350690 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:10.350695 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:10.350706 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:10.350959 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:10.351304 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:10.351307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:10.351308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:10.351464 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:10.351898 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:10.351906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:10.352162 (+ 256us) client_negotiation.cc:769] Sending connection context
0810 18:56:10.352365 (+ 203us) client_negotiation.cc:240] Negotiation successful
0810 18:56:10.352714 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":116,"threads_started":1}
I20260810 18:56:10.355051 19713 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38663
I20260810 18:56:10.355356 19713 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:10.356060 19713 heartbeater.cc:511] Master 127.30.153.190:38663 requested a full tablet report, sending...
I20260810 18:56:10.359417 19418 ts_manager.cc:194] Registered new tserver with Master: d94f3a58760c4c99b09c2cce7d283f83 (127.30.153.129:35839)
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:56:10.361317 19418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:44305
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:56:10.381599 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38663
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:10.497973 19723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:10.498216 19723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:10.498260 19723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:10.502393 19723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:10.502501 19723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:10.502560 19723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:56:10.508545 19723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:10.509596 19723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:10.510644 19723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:10.520134 19731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:10.520213 19730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:10.520149 19733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:10.520401 19723 server_base.cc:1034] running on GCE node
I20260810 18:56:10.520872 19723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:10.521529 19723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:10.522717 19723 hybrid_clock.cc:648] HybridClock initialized: now 1786388170522701 us; error 32 us; skew 500 ppm
Aug 10 18:56:10 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388170, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:10.525570 19723 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:56:10.526480 19723 webserver.cc:460] Webserver started at http://127.30.153.130:33175/ using document root <none> and password file <none>
I20260810 18:56:10.527048 19723 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:10.527141 19723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:10.527379 19723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:10.529402 19723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "2954a80f7f244c6fa724e0998da9017c"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "ba37f922e3aa6a9fadd04c4d74d6affe"
server_key_iv: "2eb35a0bdc6969cefdb6f194dcc14b04"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.529973 19723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "2954a80f7f244c6fa724e0998da9017c"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "ba37f922e3aa6a9fadd04c4d74d6affe"
server_key_iv: "2eb35a0bdc6969cefdb6f194dcc14b04"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.533776 19723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:56:10.536334 19742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.537638 19723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:56:10.537779 19723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "2954a80f7f244c6fa724e0998da9017c"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "ba37f922e3aa6a9fadd04c4d74d6affe"
server_key_iv: "2eb35a0bdc6969cefdb6f194dcc14b04"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.537895 19723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:10.557062 19723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:10.560861 19723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:10.561106 19723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:10.561710 19723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:10.562616 19723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:10.562686 19723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.562757 19723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:10.562810 19723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.572083 19723 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:39669
I20260810 18:56:10.572116 19898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:39669 every 8 connection(s)
I20260810 18:56:10.572973 19723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:56:10.578984 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 19723
I20260810 18:56:10.579113 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:56:10.579394 31334 external_mini_cluster.cc:1428] Setting key 901dd308c98040b587fa66675efc85d4
Aug 10 18:56:10 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388170, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:10.587895 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.574902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:56457 (local address 127.30.153.190:38663)
0810 18:56:10.575053 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:10.575058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:10.575963 (+ 905us) server_negotiation.cc:407] Connection header received
0810 18:56:10.577285 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:10.577288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:10.577334 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:10.577408 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:10.578674 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.579311 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.580643 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.580887 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.581002 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.583791 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:10.583794 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:10.583796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:10.583819 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:10.585505 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.586041 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.586048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.586052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.586150 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.586549 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.586555 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.586560 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.586830 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:10.586942 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:10.587483 (+ 541us) server_negotiation.cc:299] Negotiation successful
0810 18:56:10.587645 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:56:10.588624 19904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.575330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:56457)
0810 18:56:10.575808 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:10.575851 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:10.577088 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:10.577577 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:10.577584 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:10.577980 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:10.578534 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.578551 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.579439 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.579442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.580510 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.580520 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.581044 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.581048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.581226 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.581872 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:10.581889 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:10.583604 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:10.585631 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:10.585636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:10.585649 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:10.585933 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:10.586259 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:10.586261 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:10.586263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:10.586398 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:10.586953 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:10.586958 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:10.587204 (+ 246us) client_negotiation.cc:769] Sending connection context
0810 18:56:10.587367 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 18:56:10.587638 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260810 18:56:10.590157 19900 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38663
I20260810 18:56:10.590485 19900 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:10.591161 19900 heartbeater.cc:511] Master 127.30.153.190:38663 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:56:10.592595 19418 ts_manager.cc:194] Registered new tserver with Master: 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669)
I20260810 18:56:10.593276 19418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:56457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:56:10.614705 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38663
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:10.729871 19907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:10.730175 19907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:10.730237 19907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:10.734244 19907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:10.734354 19907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:10.734498 19907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:56:10.739840 19907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:10.740931 19907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:10.742038 19907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:10.751901 19917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:10.751945 19923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:10.751977 19907 server_base.cc:1034] running on GCE node
W20260810 18:56:10.752111 19926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:10.752586 19907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:10.753252 19907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:10.754485 19907 hybrid_clock.cc:648] HybridClock initialized: now 1786388170754464 us; error 36 us; skew 500 ppm
Aug 10 18:56:10 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388170, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:10.757169 19907 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:56:10.758061 19907 webserver.cc:460] Webserver started at http://127.30.153.131:42851/ using document root <none> and password file <none>
I20260810 18:56:10.758723 19907 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:10.758816 19907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:10.759029 19907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:10.761164 19907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "5de668daa1214b7fb6136ddc2f1087c9"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "85f5977edcb9d38df7df01134e1a425f"
server_key_iv: "243c1c050face4b5bc6373137ae07f0e"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.761772 19907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "5de668daa1214b7fb6136ddc2f1087c9"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "85f5977edcb9d38df7df01134e1a425f"
server_key_iv: "243c1c050face4b5bc6373137ae07f0e"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.765513 19907 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 18:56:10.768008 19933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.769208 19907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:56:10.769346 19907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "5de668daa1214b7fb6136ddc2f1087c9"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "85f5977edcb9d38df7df01134e1a425f"
server_key_iv: "243c1c050face4b5bc6373137ae07f0e"
server_key_version: "encryptionkey@0"
I20260810 18:56:10.769456 19907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:10.798275 19907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:10.801592 19907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:10.801841 19907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:10.802677 19907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:10.803632 19907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:10.803711 19907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.803794 19907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:10.803843 19907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:10.814363 19907 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:35529
I20260810 18:56:10.814594 20099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:35529 every 8 connection(s)
I20260810 18:56:10.815397 19907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:56:10.823005 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 19907
I20260810 18:56:10.823117 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:56:10.823400 31334 external_mini_cluster.cc:1428] Setting key afdfbd54f693f9a7ddf52b3964306875
Aug 10 18:56:10 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388170, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:10.830500 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.817408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:40713 (local address 127.30.153.190:38663)
0810 18:56:10.817593 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:10.817597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:10.818599 (+ 1002us) server_negotiation.cc:407] Connection header received
0810 18:56:10.820130 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:10.820134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:10.820198 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:10.820279 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:10.821555 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.822152 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.823554 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.823857 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.823995 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.826907 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:10.826910 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:10.826912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:10.826936 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:10.828369 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.828900 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.828903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.828904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.828978 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.829342 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.829345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.829347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.829535 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:10.829648 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:10.830154 (+ 506us) server_negotiation.cc:299] Negotiation successful
0810 18:56:10.830293 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 18:56:10.831267 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.817815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:40713)
0810 18:56:10.818352 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:10.818400 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:10.819884 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:10.820457 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:10.820465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:10.820861 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:10.821380 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.821395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.822304 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.822308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.823381 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.823393 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.824040 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.824044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.824266 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.825022 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:10.825058 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:10.826734 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:10.828492 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:10.828498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:10.828512 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:10.828779 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:10.829086 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:10.829089 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:10.829091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:10.829235 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:10.829648 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:10.829655 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:10.829905 (+ 250us) client_negotiation.cc:769] Sending connection context
0810 18:56:10.830094 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 18:56:10.830416 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":133,"threads_started":1}
I20260810 18:56:10.832543 20100 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38663
I20260810 18:56:10.832798 20100 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:10.833289 20100 heartbeater.cc:511] Master 127.30.153.190:38663 requested a full tablet report, sending...
I20260810 18:56:10.834388 19418 ts_manager.cc:194] Registered new tserver with Master: 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529)
I20260810 18:56:10.835076 19418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:40713
I20260810 18:56:10.838011 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:56:10.848522 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.840328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.30.153.190:38663)
0810 18:56:10.840492 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:10.840496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:10.840699 (+ 203us) server_negotiation.cc:407] Connection header received
0810 18:56:10.840785 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:10.840788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:10.840861 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:10.840950 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:10.841767 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.842402 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.843375 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.843606 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.843696 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.845163 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:10.845166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:10.845168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:10.845195 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:10.846794 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.847199 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.847202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.847203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.847282 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:10.847574 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:10.847577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:10.847579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:10.847824 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:10.847947 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:10.848156 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 18:56:10.848266 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:56:10.851670 19418 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:56:10.855101 19418 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:56:10.871012 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.865634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.30.153.129:35839)
0810 18:56:10.865954 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:10.865961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:10.866480 (+ 519us) server_negotiation.cc:407] Connection header received
0810 18:56:10.866558 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:10.866562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:10.866703 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:10.866848 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:10.867032 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.867750 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.869727 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.870272 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.870417 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.870599 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:10.870743 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 18:56:10.870853 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":120,"threads_started":1}
I20260810 18:56:10.871042 20126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.865224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:35839 (local address 127.0.0.1:40972)
0810 18:56:10.866260 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:10.866274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:10.866372 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:10.866830 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:10.866833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:10.866850 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:10.866929 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.866937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.867870 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.867877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.869607 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.869615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.870404 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.870408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.870541 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.870580 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 18:56:10.870713 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 18:56:10.870896 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":204,"thread_start_us":83,"threads_started":1}
I20260810 18:56:10.871076 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.864469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35529 (local address 127.0.0.1:41456)
0810 18:56:10.865332 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:10.865383 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:10.865526 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:10.866247 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:10.866250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:10.866270 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:10.866392 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.866402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.867950 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.867953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.869408 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.869418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.870481 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.870485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.870604 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.870626 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:56:10.870694 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:56:10.870897 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":176,"spinlock_wait_cycles":4480,"thread_start_us":125,"threads_started":1}
I20260810 18:56:10.871235 20120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.864455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:39669 (local address 127.0.0.1:58118)
0810 18:56:10.865329 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:10.865382 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:10.865526 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:10.866079 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:10.866085 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:10.866108 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:10.866283 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.866293 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.868018 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.868020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.869320 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:10.869330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.870481 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.870485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:10.870939 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.870963 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:56:10.871038 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 18:56:10.871110 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":422,"spinlock_wait_cycles":8192,"thread_start_us":83,"threads_started":1}
I20260810 18:56:10.871812 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.864880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.30.153.131:35529)
0810 18:56:10.865825 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:10.865835 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:10.865856 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:56:10.865924 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:10.865930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:10.866094 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:10.866582 (+ 488us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:10.866745 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.867828 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.869518 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.870302 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.871388 (+ 1086us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.871527 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:10.871576 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 18:56:10.871663 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":169,"threads_started":1}
I20260810 18:56:10.874595 20006 tablet_service.cc:1467] Processing CreateTablet for tablet ca1c35aefce4419795706d70191ffc23 (DEFAULT_TABLE table=test-table [id=c8e170c2a5b64d8298b99002d8dd7ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:10.874594 19620 tablet_service.cc:1467] Processing CreateTablet for tablet ca1c35aefce4419795706d70191ffc23 (DEFAULT_TABLE table=test-table [id=c8e170c2a5b64d8298b99002d8dd7ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:10.875602 19620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1c35aefce4419795706d70191ffc23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:10.875612 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1c35aefce4419795706d70191ffc23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:10.878208 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:10.864585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.30.153.130:39669)
0810 18:56:10.865567 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:10.865582 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:10.865619 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:56:10.865702 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:10.865709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:10.865926 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:10.866087 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:10.866582 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.867827 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.869448 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:10.870304 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:10.870569 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:10.870715 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:10.877926 (+ 7211us) server_negotiation.cc:299] Negotiation successful
0810 18:56:10.878051 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":164,"threads_started":1}
I20260810 18:56:10.881255 19805 tablet_service.cc:1467] Processing CreateTablet for tablet ca1c35aefce4419795706d70191ffc23 (DEFAULT_TABLE table=test-table [id=c8e170c2a5b64d8298b99002d8dd7ca2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:10.881810 20129 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Bootstrap starting.
I20260810 18:56:10.882367 19805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1c35aefce4419795706d70191ffc23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:10.884265 20129 tablet_bootstrap.cc:654] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:10.885023 20129 log.cc:824] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:10.886945 20129 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: No bootstrap required, opened a new log
I20260810 18:56:10.887152 20129 ts_tablet_manager.cc:1397] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:56:10.889839 20132 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Bootstrap starting.
I20260810 18:56:10.890326 20130 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Bootstrap starting.
I20260810 18:56:10.891939 20129 raft_consensus.cc:348] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:10.892256 20129 raft_consensus.cc:374] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:10.892324 20129 raft_consensus.cc:729] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d94f3a58760c4c99b09c2cce7d283f83, State: Initialized, Role: FOLLOWER
I20260810 18:56:10.892827 20132 tablet_bootstrap.cc:654] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:10.892848 20130 tablet_bootstrap.cc:654] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:10.892856 20129 consensus_queue.cc:260] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:10.893621 19713 heartbeater.cc:503] Master 127.30.153.190:38663 was elected leader, sending a full tablet report...
I20260810 18:56:10.893772 20130 log.cc:824] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:10.893826 20132 log.cc:824] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:10.893959 20129 ts_tablet_manager.cc:1428] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 18:56:10.895988 20132 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: No bootstrap required, opened a new log
I20260810 18:56:10.896196 20132 ts_tablet_manager.cc:1397] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:56:10.896260 20130 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: No bootstrap required, opened a new log
I20260810 18:56:10.896440 20130 ts_tablet_manager.cc:1397] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260810 18:56:10.899607 20132 raft_consensus.cc:348] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:10.899827 20132 raft_consensus.cc:374] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:10.899863 20132 raft_consensus.cc:729] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2954a80f7f244c6fa724e0998da9017c, State: Initialized, Role: FOLLOWER
I20260810 18:56:10.899827 20130 raft_consensus.cc:348] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:10.900098 20130 raft_consensus.cc:374] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:10.900161 20130 raft_consensus.cc:729] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de668daa1214b7fb6136ddc2f1087c9, State: Initialized, Role: FOLLOWER
I20260810 18:56:10.900377 20132 consensus_queue.cc:260] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:10.900552 20130 consensus_queue.cc:260] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:10.901093 19900 heartbeater.cc:503] Master 127.30.153.190:38663 was elected leader, sending a full tablet report...
I20260810 18:56:10.901204 20100 heartbeater.cc:503] Master 127.30.153.190:38663 was elected leader, sending a full tablet report...
I20260810 18:56:10.901549 20130 ts_tablet_manager.cc:1428] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:56:10.902542 20132 ts_tablet_manager.cc:1428] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:56:11.043946 20133 raft_consensus.cc:482] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:11.044226 20133 raft_consensus.cc:504] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:11.045497 20133 leader_election.cc:288] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669), 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529)
I20260810 18:56:11.049827 20125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.045925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:34977 (local address 127.30.153.131:35529)
0810 18:56:11.046078 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:11.046083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:11.046653 (+ 570us) server_negotiation.cc:407] Connection header received
0810 18:56:11.046718 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:11.046721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:11.046776 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:11.046878 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:11.047172 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.047683 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.048814 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.049221 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.049373 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.049431 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:11.049625 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 18:56:11.049701 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:56:11.049829 20123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.045887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:36605 (local address 127.30.153.130:39669)
0810 18:56:11.046063 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:11.046068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:11.046258 (+ 190us) server_negotiation.cc:407] Connection header received
0810 18:56:11.046488 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:11.046493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:11.046564 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:11.046652 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:11.047029 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.047605 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.048819 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.049209 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.049332 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.049389 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:11.049589 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 18:56:11.049661 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:56:11.049881 20140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.045772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:39669 (local address 127.30.153.129:36605)
0810 18:56:11.046131 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:11.046147 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:11.046320 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:11.046706 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:11.046709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:11.046731 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:11.046904 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.046911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.047729 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.047733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.048679 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.048688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.049340 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.049344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.049449 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.049477 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:56:11.049547 (+ 70us) client_negotiation.cc:240] Negotiation successful
0810 18:56:11.049644 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":63,"thread_start_us":147,"threads_started":1}
I20260810 18:56:11.049860 20141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.045774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35529 (local address 127.30.153.129:34977)
0810 18:56:11.046488 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:11.046502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:11.046620 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:11.046896 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:11.046898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:11.046914 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:11.047047 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.047054 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.047798 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.047801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.048675 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.048684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.049381 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.049385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.049478 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.049495 (+ 17us) client_negotiation.cc:769] Sending connection context
0810 18:56:11.049588 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:56:11.049664 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":331,"thread_start_us":130,"threads_started":1}
I20260810 18:56:11.050724 19834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1c35aefce4419795706d70191ffc23" candidate_uuid: "d94f3a58760c4c99b09c2cce7d283f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2954a80f7f244c6fa724e0998da9017c" is_pre_election: true
I20260810 18:56:11.050765 20033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1c35aefce4419795706d70191ffc23" candidate_uuid: "d94f3a58760c4c99b09c2cce7d283f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de668daa1214b7fb6136ddc2f1087c9" is_pre_election: true
I20260810 18:56:11.051059 20033 raft_consensus.cc:2454] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d94f3a58760c4c99b09c2cce7d283f83 in term 0.
I20260810 18:56:11.051066 19834 raft_consensus.cc:2454] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d94f3a58760c4c99b09c2cce7d283f83 in term 0.
I20260810 18:56:11.051563 19540 leader_election.cc:302] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de668daa1214b7fb6136ddc2f1087c9, d94f3a58760c4c99b09c2cce7d283f83; no voters:
I20260810 18:56:11.051849 20133 raft_consensus.cc:2781] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:56:11.051939 20133 raft_consensus.cc:482] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:56:11.051988 20133 raft_consensus.cc:3037] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:11.053045 20133 raft_consensus.cc:504] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:11.053449 20133 leader_election.cc:288] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 1 election: Requested vote from peers 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669), 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529)
I20260810 18:56:11.053787 19834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1c35aefce4419795706d70191ffc23" candidate_uuid: "d94f3a58760c4c99b09c2cce7d283f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2954a80f7f244c6fa724e0998da9017c"
I20260810 18:56:11.053846 20033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1c35aefce4419795706d70191ffc23" candidate_uuid: "d94f3a58760c4c99b09c2cce7d283f83" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5de668daa1214b7fb6136ddc2f1087c9"
I20260810 18:56:11.053932 19834 raft_consensus.cc:3037] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:11.053984 20033 raft_consensus.cc:3037] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:11.055079 20033 raft_consensus.cc:2454] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d94f3a58760c4c99b09c2cce7d283f83 in term 1.
I20260810 18:56:11.055089 19834 raft_consensus.cc:2454] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d94f3a58760c4c99b09c2cce7d283f83 in term 1.
I20260810 18:56:11.055366 19540 leader_election.cc:302] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de668daa1214b7fb6136ddc2f1087c9, d94f3a58760c4c99b09c2cce7d283f83; no voters:
I20260810 18:56:11.055545 20133 raft_consensus.cc:2781] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:11.055796 20133 raft_consensus.cc:686] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 LEADER]: Becoming Leader. State: Replica: d94f3a58760c4c99b09c2cce7d283f83, State: Running, Role: LEADER
I20260810 18:56:11.056118 20133 consensus_queue.cc:237] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:11.059414 19418 catalog_manager.cc:5495] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 reported cstate change: term changed from 0 to 1, leader changed from <none> to d94f3a58760c4c99b09c2cce7d283f83 (127.30.153.129). New cstate: current_term: 1 leader_uuid: "d94f3a58760c4c99b09c2cce7d283f83" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } health_report { overall_health: HEALTHY } } }
W20260810 18:56:11.066587 20101 tablet.cc:2369] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:56:11.074156 19902 tablet.cc:2369] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:56:11.089850 19715 tablet.cc:2369] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:56:11.107987 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.104165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.30.153.129:35839)
0810 18:56:11.104341 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:11.104348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:11.104369 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:56:11.104466 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:11.104468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:11.104527 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:11.104665 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:56:11.105007 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.105532 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.106460 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.106693 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.106774 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.107067 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:56:11.107497 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:56:11.107593 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:11.107736 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 18:56:11.107792 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 18:56:11.114284 20033 raft_consensus.cc:1261] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 FOLLOWER]: Refusing update from remote peer d94f3a58760c4c99b09c2cce7d283f83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:11.114338 19834 raft_consensus.cc:1261] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 1 FOLLOWER]: Refusing update from remote peer d94f3a58760c4c99b09c2cce7d283f83: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:11.115116 20142 consensus_queue.cc:1035] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:11.115327 20133 consensus_queue.cc:1035] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:11.125010 20152 mvcc.cc:205] Tried to move back new op lower bound from 7317045948876939264 to 7317045948648419328. Current Snapshot: MvccSnapshot[applied={T|T < 7317045948876939264 or (T in {7317045948876939264})}]
I20260810 18:56:11.135699 19414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:11.135912 19414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:11.138361 19414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Sending DeleteTablet for 3 replicas of tablet ca1c35aefce4419795706d70191ffc23
I20260810 18:56:11.139235 19805 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1c35aefce4419795706d70191ffc23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:11 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:58118
I20260810 18:56:11.139353 20006 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1c35aefce4419795706d70191ffc23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:11 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:41456
I20260810 18:56:11.139626 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 19516
I20260810 18:56:11.140014 20163 tablet_replica.cc:330] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: stopping tablet replica
I20260810 18:56:11.140379 20163 raft_consensus.cc:2229] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:56:11.140758 20163 raft_consensus.cc:2258] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:11.144204 20164 tablet_replica.cc:330] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: stopping tablet replica
I20260810 18:56:11.144516 20164 raft_consensus.cc:2229] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:56:11.144779 20164 raft_consensus.cc:2258] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:11.146467 20163 ts_tablet_manager.cc:1905] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:11.147312 20164 ts_tablet_manager.cc:1905] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:11.149691 20163 ts_tablet_manager.cc:1918] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:11.149803 20163 log.cc:1196] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/ca1c35aefce4419795706d70191ffc23
I20260810 18:56:11.150158 20163 ts_tablet_manager.cc:1939] T ca1c35aefce4419795706d70191ffc23 P 2954a80f7f244c6fa724e0998da9017c: Deleting consensus metadata
I20260810 18:56:11.150274 20164 ts_tablet_manager.cc:1918] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:11.150367 20164 log.cc:1196] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/ca1c35aefce4419795706d70191ffc23
I20260810 18:56:11.150744 20164 ts_tablet_manager.cc:1939] T ca1c35aefce4419795706d70191ffc23 P 5de668daa1214b7fb6136ddc2f1087c9: Deleting consensus metadata
I20260810 18:56:11.151813 19394 catalog_manager.cc:4841] TS 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669): tablet ca1c35aefce4419795706d70191ffc23 (table test-table [id=c8e170c2a5b64d8298b99002d8dd7ca2]) successfully deleted
I20260810 18:56:11.151813 19398 catalog_manager.cc:4841] TS 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529): tablet ca1c35aefce4419795706d70191ffc23 (table test-table [id=c8e170c2a5b64d8298b99002d8dd7ca2]) successfully deleted
W20260810 18:56:11.159399 19394 connection.cc:535] client connection to 127.30.153.129:35839 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260810 18:56:11.159664 19394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:56:11.160367 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 19723
I20260810 18:56:11.166136 20126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.160529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:35839 (local address 127.0.0.1:41002)
0810 18:56:11.165924 (+ 5395us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:11.166021 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:35839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5262}
W20260810 18:56:11.175556 19394 catalog_manager.cc:4584] TS d94f3a58760c4c99b09c2cce7d283f83 (127.30.153.129:35839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca1c35aefce4419795706d70191ffc23: Network error: Client connection negotiation failed: client connection to 127.30.153.129:35839: connect: Connection refused (error 111)
I20260810 18:56:11.182698 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 19907
I20260810 18:56:11.196813 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 19365
I20260810 18:56:11.209141 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38663
--webserver_interface=127.30.153.190
--webserver_port=39095
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:11.320494 20166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:11.320815 20166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:11.320899 20166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:11.324891 20166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:11.324991 20166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:11.325029 20166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:11.325103 20166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:11.325141 20166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:11.330389 20166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:11.331475 20166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:11.332515 20166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:56:11.346963 20166 server_base.cc:1034] running on GCE node
W20260810 18:56:11.346953 20178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:11.346933 20180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:11.347150 20177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:11.347775 20166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:11.348886 20166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:11.350080 20166 hybrid_clock.cc:648] HybridClock initialized: now 1786388171350070 us; error 32 us; skew 500 ppm
Aug 10 18:56:11 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388171, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:11.353149 20166 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:56:11.354097 20166 webserver.cc:460] Webserver started at http://127.30.153.190:39095/ using document root <none> and password file <none>
I20260810 18:56:11.354805 20166 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:11.354925 20166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:11.359206 20166 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:56:11.362084 20190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:11.363456 20166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:56:11.363605 20166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "54f600d408ad4aa2a38ced4bb38749b0"
format_stamp: "Formatted at 2026-08-10 18:56:09 on dist-test-slave-68xk"
server_key: "dd94ad69273798035910d33955c9db5c"
server_key_iv: "6caee141edbf2a8a2f7263021d9c6c84"
server_key_version: "encryptionkey@0"
I20260810 18:56:11.364234 20166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:11.389762 20166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:11.393112 20166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:11.393386 20166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:11.401527 20166 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:38663
I20260810 18:56:11.401834 20272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:38663 every 8 connection(s)
I20260810 18:56:11.402894 20166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:56:11.406235 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 20166
I20260810 18:56:11.406801 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:35839
--local_ip_for_outbound_sockets=127.30.153.129
--tserver_master_addrs=127.30.153.190:38663
--webserver_port=37481
--webserver_interface=127.30.153.129
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:11.407821 20275 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:56:11.409555 20275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Bootstrap starting.
I20260810 18:56:11.434882 20275 log.cc:824] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:11.446159 20275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:56:11.446631 20275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Bootstrap complete.
I20260810 18:56:11.452250 20275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:11.452869 20275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f600d408ad4aa2a38ced4bb38749b0, State: Initialized, Role: FOLLOWER
I20260810 18:56:11.453418 20275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:11.453567 20275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:56:11.453648 20275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:56:11.453755 20275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:56:11.456375 20275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:11.456748 20275 leader_election.cc:302] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54f600d408ad4aa2a38ced4bb38749b0; no voters:
I20260810 18:56:11.457111 20275 leader_election.cc:288] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:56:11.457283 20281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:56:11.457594 20281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 2 LEADER]: Becoming Leader. State: Replica: 54f600d408ad4aa2a38ced4bb38749b0, State: Running, Role: LEADER
I20260810 18:56:11.457928 20281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:11.458453 20275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:11.459721 20285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "54f600d408ad4aa2a38ced4bb38749b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } } }
I20260810 18:56:11.459721 20288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54f600d408ad4aa2a38ced4bb38749b0. Latest consensus state: current_term: 2 leader_uuid: "54f600d408ad4aa2a38ced4bb38749b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } } }
I20260810 18:56:11.459885 20288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:11.459885 20285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:11.464395 20307 catalog_manager.cc:1259] Loaded cluster ID: 6e07aac6588d4b65a8a5b22475737136
I20260810 18:56:11.464464 20307 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: loading cluster ID for follower catalog manager: success
W20260810 18:56:11.465530 20307 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260810 18:56:11.465613 20307 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260810 18:56:11.465703 20300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:56:11.468434 20300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:56:11.468995 20300 catalog_manager.cc:1259] Loaded cluster ID: 6e07aac6588d4b65a8a5b22475737136
I20260810 18:56:11.469060 20300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:56:11.470250 20300 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 18:56:11.475839 20300 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 18:56:11.481671 20300 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:56:11.482559 20300 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Loaded TSK: 0
I20260810 18:56:11.483858 20300 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:56:11.540208 20279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:11.540495 20279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:11.540570 20279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:11.544842 20279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:11.544915 20279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:11.544996 20279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:56:11.550379 20279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:35839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=37481
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:11.551442 20279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:11.552486 20279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:11.563678 20319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:11.563720 20318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:11.563810 20279 server_base.cc:1034] running on GCE node
W20260810 18:56:11.564026 20321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:11.564422 20279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:11.565125 20279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:11.566362 20279 hybrid_clock.cc:648] HybridClock initialized: now 1786388171566340 us; error 43 us; skew 500 ppm
Aug 10 18:56:11 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388171, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:11.569734 20279 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:56:11.570806 20279 webserver.cc:460] Webserver started at http://127.30.153.129:37481/ using document root <none> and password file <none>
I20260810 18:56:11.571393 20279 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:11.571516 20279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:11.575582 20279 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:56:11.577852 20331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:11.579217 20279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:56:11.579356 20279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "d94f3a58760c4c99b09c2cce7d283f83"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "d16afb6d674cd99eba6bffa668cc3e6f"
server_key_iv: "2414767960c999886bdf016e405216d5"
server_key_version: "encryptionkey@0"
I20260810 18:56:11.579906 20279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:11.608958 20279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:11.612538 20279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:11.612777 20279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:11.613530 20279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:11.614933 20340 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 18:56:11.619815 20279 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:56:11.619884 20279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 18:56:11.619966 20279 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:56:11.621654 20279 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:56:11.621701 20279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 18:56:11.621999 20340 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Bootstrap starting.
I20260810 18:56:11.632599 20279 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:35839
I20260810 18:56:11.632656 20500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:35839 every 8 connection(s)
I20260810 18:56:11.633548 20279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:56:11.636211 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 20279
I20260810 18:56:11.636778 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:39669
--local_ip_for_outbound_sockets=127.30.153.130
--tserver_master_addrs=127.30.153.190:38663
--webserver_port=33175
--webserver_interface=127.30.153.130
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:11.644348 20340 log.cc:824] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:56:11 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388171, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:11.654726 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.636111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:51181 (local address 127.30.153.190:38663)
0810 18:56:11.636814 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:11.636835 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:11.637651 (+ 816us) server_negotiation.cc:407] Connection header received
0810 18:56:11.639325 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:11.639346 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:11.639806 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:11.640220 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:11.641946 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.643180 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.644659 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.645417 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.645595 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.648786 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:11.648792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:11.648806 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:11.648839 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:11.651007 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:11.651674 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:11.651680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:11.651684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:11.651826 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:11.652239 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:11.652243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:11.652245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:11.652646 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:11.652817 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:11.653410 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 18:56:11.653715 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":162,"threads_started":1}
I20260810 18:56:11.654834 20505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.636074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:51181)
0810 18:56:11.637408 (+ 1334us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:11.637483 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:11.638788 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:11.640322 (+ 1534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:11.640332 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:11.640882 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:11.641722 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.641746 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.643383 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.643386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.644448 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.644459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.645721 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.645725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.645915 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.646820 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:11.646839 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:11.648539 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:11.651169 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:11.651175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:11.651189 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:11.651492 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:11.651947 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:11.651950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:11.651952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:11.652121 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:11.652786 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:11.652794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:11.653060 (+ 266us) client_negotiation.cc:769] Sending connection context
0810 18:56:11.653277 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 18:56:11.653664 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"thread_start_us":162,"threads_started":1}
I20260810 18:56:11.655593 20340 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:56:11.656090 20340 tablet_bootstrap.cc:492] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Bootstrap complete.
I20260810 18:56:11.657567 20501 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38663
I20260810 18:56:11.657855 20501 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:11.657934 20340 ts_tablet_manager.cc:1397] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Time spent bootstrapping tablet: real 0.036s user 0.028s sys 0.004s
I20260810 18:56:11.658635 20501 heartbeater.cc:511] Master 127.30.153.190:38663 requested a full tablet report, sending...
I20260810 18:56:11.661445 20218 ts_manager.cc:194] Registered new tserver with Master: d94f3a58760c4c99b09c2cce7d283f83 (127.30.153.129:35839)
I20260810 18:56:11.661183 20340 raft_consensus.cc:348] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:11.661743 20340 raft_consensus.cc:729] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d94f3a58760c4c99b09c2cce7d283f83, State: Initialized, Role: FOLLOWER
I20260810 18:56:11.662284 20340 consensus_queue.cc:260] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:11.663482 20340 ts_tablet_manager.cc:1428] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260810 18:56:11.665616 20218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:51181
I20260810 18:56:11.668548 20501 heartbeater.cc:503] Master 127.30.153.190:38663 was elected leader, sending a full tablet report...
Aug 10 18:56:11 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388171, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:56:11.677111 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.664957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54762 (local address 127.30.153.129:35839)
0810 18:56:11.666551 (+ 1594us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:11.666562 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:11.666583 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:56:11.666644 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:11.666647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:11.666874 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:11.667026 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:11.667855 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.669090 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.670076 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.670689 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.670817 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.673274 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:11.673285 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:11.673299 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:11.673333 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:11.675100 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:11.675621 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:11.675626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:11.675628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:11.675710 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:11.676017 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:11.676022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:11.676024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:11.676294 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:11.676436 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:11.676785 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 18:56:11.676953 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1452,"thread_start_us":148,"threads_started":1}
I20260810 18:56:11.677111 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.664857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:35839 (local address 127.0.0.1:54762)
0810 18:56:11.665559 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:11.665616 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:11.665767 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:11.667065 (+ 1298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:11.667072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:11.667557 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:11.667708 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.667717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.669226 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.669230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.669926 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.669936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.670832 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.670836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.670972 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.671614 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:11.671627 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:11.673116 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:11.675231 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:11.675241 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:11.675246 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:11.675484 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:11.675825 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:11.675829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:11.675831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:11.675920 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:11.676426 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:11.676431 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:11.676556 (+ 125us) client_negotiation.cc:769] Sending connection context
0810 18:56:11.676702 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 18:56:11.676918 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":118,"threads_started":1}
I20260810 18:56:11.678166 20404 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1c35aefce4419795706d70191ffc23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:11 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:54762
I20260810 18:56:11.679133 20513 tablet_replica.cc:330] stopping tablet replica
I20260810 18:56:11.679450 20513 raft_consensus.cc:2229] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:56:11.679711 20513 raft_consensus.cc:2258] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:11.680699 20513 ts_tablet_manager.cc:1905] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:11.684578 20513 ts_tablet_manager.cc:1918] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:11.684661 20513 log.cc:1196] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/ca1c35aefce4419795706d70191ffc23
I20260810 18:56:11.684954 20513 ts_tablet_manager.cc:1939] T ca1c35aefce4419795706d70191ffc23 P d94f3a58760c4c99b09c2cce7d283f83: Deleting consensus metadata
I20260810 18:56:11.686682 20191 catalog_manager.cc:4841] TS d94f3a58760c4c99b09c2cce7d283f83 (127.30.153.129:35839): tablet ca1c35aefce4419795706d70191ffc23 (table test-table [id=c8e170c2a5b64d8298b99002d8dd7ca2]) successfully deleted
W20260810 18:56:11.774546 20506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:11.774829 20506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:11.774874 20506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:11.779047 20506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:11.779119 20506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:11.779179 20506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:56:11.784687 20506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:39669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=33175
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:11.785882 20506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:11.787058 20506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:11.797118 20520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:11.797152 20519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:11.797123 20526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:11.797254 20506 server_base.cc:1034] running on GCE node
I20260810 18:56:11.798626 20506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:11.799229 20506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:11.800395 20506 hybrid_clock.cc:648] HybridClock initialized: now 1786388171800384 us; error 28 us; skew 500 ppm
Aug 10 18:56:11 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388171, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:11.803158 20506 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:56:11.804008 20506 webserver.cc:460] Webserver started at http://127.30.153.130:33175/ using document root <none> and password file <none>
I20260810 18:56:11.804632 20506 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:11.804715 20506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:11.808854 20506 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:56:11.810963 20539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:11.812217 20506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:56:11.812328 20506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "2954a80f7f244c6fa724e0998da9017c"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "ba37f922e3aa6a9fadd04c4d74d6affe"
server_key_iv: "2eb35a0bdc6969cefdb6f194dcc14b04"
server_key_version: "encryptionkey@0"
I20260810 18:56:11.812848 20506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:11.865070 20506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:11.868788 20506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:11.869009 20506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:11.869649 20506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:11.870572 20506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:11.870620 20506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:11.870693 20506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:11.870754 20506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:11.879582 20506 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:39669
I20260810 18:56:11.879618 20715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:39669 every 8 connection(s)
I20260810 18:56:11.880621 20506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:56:11.888839 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 20506
I20260810 18:56:11.889367 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:35529
--local_ip_for_outbound_sockets=127.30.153.131
--tserver_master_addrs=127.30.153.190:38663
--webserver_port=42851
--webserver_interface=127.30.153.131
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:56:11 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388171, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:11.895601 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.882502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:48655 (local address 127.30.153.190:38663)
0810 18:56:11.882659 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:11.882665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:11.883713 (+ 1048us) server_negotiation.cc:407] Connection header received
0810 18:56:11.885197 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:11.885201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:11.885277 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:11.885377 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:11.886646 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.887194 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.888375 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.888616 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.888735 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.891875 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:11.891879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:11.891881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:11.891907 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:11.893366 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:11.893951 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:11.893954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:11.893956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:11.894039 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:11.894441 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:11.894447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:11.894453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:11.894636 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:11.894751 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:11.895277 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 18:56:11.895423 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:56:11.896520 20719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:11.883027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:48655)
0810 18:56:11.883530 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:11.883574 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:11.884916 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:11.885570 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:11.885577 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:11.885949 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:11.886491 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.886507 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.887344 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.887354 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.888239 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:11.888249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:11.888786 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:11.888790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:11.888958 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:11.889727 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:11.889744 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:11.891669 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:11.893479 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:11.893484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:11.893496 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:11.893820 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:11.894136 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:11.894142 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:11.894145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:11.894301 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:11.894755 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:11.894764 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:11.895020 (+ 256us) client_negotiation.cc:769] Sending connection context
0810 18:56:11.895208 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 18:56:11.895539 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":119,"threads_started":1}
I20260810 18:56:11.897769 20716 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38663
I20260810 18:56:11.898032 20716 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:11.898604 20716 heartbeater.cc:511] Master 127.30.153.190:38663 requested a full tablet report, sending...
I20260810 18:56:11.899731 20218 ts_manager.cc:194] Registered new tserver with Master: 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669)
I20260810 18:56:11.900631 20218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:48655
W20260810 18:56:12.006376 20721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:12.006779 20721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:12.006875 20721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:12.010950 20721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:12.011015 20721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:12.011075 20721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:56:12.016548 20721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:35529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=42851
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:12.017711 20721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:12.018711 20721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:12.028292 20728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:12.028342 20727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:12.028316 20731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:12.028578 20721 server_base.cc:1034] running on GCE node
I20260810 18:56:12.029146 20721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:12.029728 20721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:12.030884 20721 hybrid_clock.cc:648] HybridClock initialized: now 1786388172030854 us; error 43 us; skew 500 ppm
Aug 10 18:56:12 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388172, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:12.033586 20721 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:56:12.034474 20721 webserver.cc:460] Webserver started at http://127.30.153.131:42851/ using document root <none> and password file <none>
I20260810 18:56:12.035023 20721 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:12.035097 20721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:12.038931 20721 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:56:12.041008 20740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:12.042205 20721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:56:12.042470 20721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "5de668daa1214b7fb6136ddc2f1087c9"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "85f5977edcb9d38df7df01134e1a425f"
server_key_iv: "243c1c050face4b5bc6373137ae07f0e"
server_key_version: "encryptionkey@0"
I20260810 18:56:12.042984 20721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:12.084920 20721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:12.088502 20721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:12.088725 20721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:12.089382 20721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:12.090313 20721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:12.090365 20721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:12.090458 20721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:12.090502 20721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:12.099632 20721 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:35529
I20260810 18:56:12.099659 20908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:35529 every 8 connection(s)
I20260810 18:56:12.100735 20721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:56:12.108577 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 20721
Aug 10 18:56:12 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388172, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:12.115577 20503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.102606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:35363 (local address 127.30.153.190:38663)
0810 18:56:12.102749 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:12.102753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:12.103578 (+ 825us) server_negotiation.cc:407] Connection header received
0810 18:56:12.104995 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:12.104999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:12.105077 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:12.105199 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:12.106509 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.107146 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.108402 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.108751 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.108866 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.111839 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:12.111843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:12.111845 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:12.111866 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:12.113470 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:12.114025 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:12.114028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:12.114029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:12.114106 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:12.114481 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:12.114485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:12.114486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:12.114663 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:12.114780 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:12.115282 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 18:56:12.115404 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:56:12.116499 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.102977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:35363)
0810 18:56:12.103425 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.103469 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:12.104757 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:12.105396 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:12.105403 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:12.105816 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:12.106319 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.106333 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.107274 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.107277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.108269 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.108280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.108930 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.108933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.109176 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.109901 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:12.109943 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:12.111656 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:12.113600 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:12.113606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:12.113620 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:12.113899 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:12.114203 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:12.114206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:12.114208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:12.114350 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:12.114781 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:12.114788 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:12.115025 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:56:12.115211 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 18:56:12.115526 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":116,"threads_started":1}
I20260810 18:56:12.117733 20909 heartbeater.cc:347] Connected to a master server at 127.30.153.190:38663
I20260810 18:56:12.117936 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.109360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.30.153.190:38663)
0810 18:56:12.109767 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:12.109771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:12.109866 (+ 95us) server_negotiation.cc:407] Connection header received
0810 18:56:12.109973 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:12.109976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:12.110027 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:12.110131 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:12.111018 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.111608 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.112649 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.113031 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.113153 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.114512 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:12.114515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:12.114518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:12.114544 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:12.116255 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:12.116702 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:12.116705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:12.116707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:12.116798 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:12.117084 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:12.117088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:12.117090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:12.117299 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:12.117505 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:12.117642 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 18:56:12.117754 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":104,"threads_started":1}
I20260810 18:56:12.118016 20909 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:12.118616 20909 heartbeater.cc:511] Master 127.30.153.190:38663 requested a full tablet report, sending...
I20260810 18:56:12.119728 20216 ts_manager.cc:194] Registered new tserver with Master: 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529)
I20260810 18:56:12.120291 20216 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:35363
I20260810 18:56:12.131379 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:56:12.135993 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.132006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44726 (local address 127.30.153.190:38663)
0810 18:56:12.132151 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:12.132158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:12.132362 (+ 204us) server_negotiation.cc:407] Connection header received
0810 18:56:12.132569 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:12.132571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:12.132663 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:12.132770 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:56:12.133028 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.133527 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.134466 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.134778 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.134894 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.135225 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:56:12.135597 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:56:12.135729 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:12.135801 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 18:56:12.135857 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:56:12.136636 20216 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:56:12.139925 20216 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:56:12.147787 20404 tablet_service.cc:1467] Processing CreateTablet for tablet c8866182f8414adcbe54291e219b25b4 (DEFAULT_TABLE table=test-table [id=4a3a32c28c5e4bdcbe864efb629711ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:12.148406 20404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8866182f8414adcbe54291e219b25b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:12.151441 20340 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Bootstrap starting.
I20260810 18:56:12.152416 20340 tablet_bootstrap.cc:654] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:12.153517 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.146881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:39669 (local address 127.0.0.1:55114)
0810 18:56:12.147416 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.147431 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:12.147522 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:12.149205 (+ 1683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:12.149208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:12.149227 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:12.149308 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.149314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.150576 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.150580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.151523 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.151535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.153041 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.153043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.153108 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.153172 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 18:56:12.153243 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 18:56:12.153330 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":69,"thread_start_us":111,"threads_started":1}
I20260810 18:56:12.153518 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.147105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55114 (local address 127.30.153.130:39669)
0810 18:56:12.148538 (+ 1433us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:12.148553 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:12.148576 (+ 23us) server_negotiation.cc:407] Connection header received
0810 18:56:12.148651 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:12.148656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:12.148854 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:12.149028 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:12.149432 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.150396 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.152006 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.152910 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.153050 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.153209 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:12.153278 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 18:56:12.153371 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1310,"thread_start_us":155,"threads_started":1}
I20260810 18:56:12.153877 20340 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: No bootstrap required, opened a new log
I20260810 18:56:12.153976 20340 ts_tablet_manager.cc:1397] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:56:12.154482 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.146571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35529 (local address 127.0.0.1:38194)
0810 18:56:12.147018 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.147035 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:12.147115 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:12.148265 (+ 1150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:12.148268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:12.148288 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:12.148389 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.148398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.150136 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.150139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.151517 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.151530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.154031 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.154035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.154143 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.154168 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:56:12.154259 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:56:12.154338 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":11}
I20260810 18:56:12.154537 20925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.146790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38194 (local address 127.30.153.131:35529)
0810 18:56:12.147758 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:12.147772 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:12.147800 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:56:12.147878 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:12.147883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:12.148103 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:12.148276 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:12.148536 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.149978 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.151674 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.153844 (+ 2170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.154046 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.154174 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:12.154288 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 18:56:12.154378 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":186,"threads_started":1}
I20260810 18:56:12.154507 20340 raft_consensus.cc:348] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.154641 20340 raft_consensus.cc:374] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:12.154693 20340 raft_consensus.cc:729] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d94f3a58760c4c99b09c2cce7d283f83, State: Initialized, Role: FOLLOWER
I20260810 18:56:12.157173 20617 tablet_service.cc:1467] Processing CreateTablet for tablet c8866182f8414adcbe54291e219b25b4 (DEFAULT_TABLE table=test-table [id=4a3a32c28c5e4bdcbe864efb629711ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:12.157784 20808 tablet_service.cc:1467] Processing CreateTablet for tablet c8866182f8414adcbe54291e219b25b4 (DEFAULT_TABLE table=test-table [id=4a3a32c28c5e4bdcbe864efb629711ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:12.158375 20617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8866182f8414adcbe54291e219b25b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:12.158825 20808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8866182f8414adcbe54291e219b25b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:12.162564 20340 consensus_queue.cc:260] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.162922 20340 ts_tablet_manager.cc:1428] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20260810 18:56:12.167241 20930 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Bootstrap starting.
I20260810 18:56:12.167605 20934 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: Bootstrap starting.
I20260810 18:56:12.170673 20930 tablet_bootstrap.cc:654] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:12.170672 20934 tablet_bootstrap.cc:654] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:12.171626 20934 log.cc:824] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:12.171626 20930 log.cc:824] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:12.173799 20930 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: No bootstrap required, opened a new log
I20260810 18:56:12.174009 20930 ts_tablet_manager.cc:1397] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:56:12.174036 20934 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: No bootstrap required, opened a new log
I20260810 18:56:12.174199 20934 ts_tablet_manager.cc:1397] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:56:12.178087 20934 raft_consensus.cc:348] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.178087 20930 raft_consensus.cc:348] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.178377 20930 raft_consensus.cc:374] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:12.178377 20934 raft_consensus.cc:374] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:12.178467 20934 raft_consensus.cc:729] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2954a80f7f244c6fa724e0998da9017c, State: Initialized, Role: FOLLOWER
I20260810 18:56:12.178467 20930 raft_consensus.cc:729] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5de668daa1214b7fb6136ddc2f1087c9, State: Initialized, Role: FOLLOWER
I20260810 18:56:12.178961 20930 consensus_queue.cc:260] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.178961 20934 consensus_queue.cc:260] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.179656 20909 heartbeater.cc:503] Master 127.30.153.190:38663 was elected leader, sending a full tablet report...
I20260810 18:56:12.179888 20716 heartbeater.cc:503] Master 127.30.153.190:38663 was elected leader, sending a full tablet report...
I20260810 18:56:12.180050 20930 ts_tablet_manager.cc:1428] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:56:12.181066 20934 ts_tablet_manager.cc:1428] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 18:56:12.285547 20941 raft_consensus.cc:482] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:12.285770 20941 raft_consensus.cc:504] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.286967 20941 leader_election.cc:288] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669), d94f3a58760c4c99b09c2cce7d283f83 (127.30.153.129:35839)
I20260810 18:56:12.290771 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.287425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:42543 (local address 127.30.153.130:39669)
0810 18:56:12.287630 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:12.287636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:12.287655 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:56:12.287720 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:12.287726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:12.287790 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:12.287880 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:12.288211 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.288719 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.289785 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.290185 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.290317 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.290368 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:12.290586 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 18:56:12.290653 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 18:56:12.290771 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.287312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:39669 (local address 127.30.153.131:42543)
0810 18:56:12.287450 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.287468 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:12.287585 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:12.287924 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:12.287927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:12.287945 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:12.288070 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.288082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.288839 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.288843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.289650 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.289659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.290310 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.290314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.290450 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.290468 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 18:56:12.290560 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 18:56:12.290635 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":25}
I20260810 18:56:12.291390 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.287585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:35839 (local address 127.30.153.131:36621)
0810 18:56:12.287989 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.288003 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:12.288151 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:12.288462 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:12.288465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:12.288480 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:12.288600 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.288607 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.289360 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.289365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.290161 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:12.290170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.291029 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.291034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:12.291123 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.291146 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:56:12.291234 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 18:56:12.291286 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":1,"thread_start_us":161,"threads_started":1}
I20260810 18:56:12.291479 20943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.287707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:36621 (local address 127.30.153.129:35839)
0810 18:56:12.288019 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:12.288027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:12.288108 (+ 81us) server_negotiation.cc:407] Connection header received
0810 18:56:12.288266 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:12.288271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:12.288335 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:12.288452 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:12.288732 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.289238 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.290296 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.290812 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.290974 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.291124 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:12.291270 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 18:56:12.291350 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":114,"threads_started":1}
I20260810 18:56:12.291529 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8866182f8414adcbe54291e219b25b4" candidate_uuid: "5de668daa1214b7fb6136ddc2f1087c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2954a80f7f244c6fa724e0998da9017c" is_pre_election: true
I20260810 18:56:12.291884 20646 raft_consensus.cc:2454] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5de668daa1214b7fb6136ddc2f1087c9 in term 0.
I20260810 18:56:12.292096 20435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8866182f8414adcbe54291e219b25b4" candidate_uuid: "5de668daa1214b7fb6136ddc2f1087c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d94f3a58760c4c99b09c2cce7d283f83" is_pre_election: true
I20260810 18:56:12.292408 20435 raft_consensus.cc:2454] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5de668daa1214b7fb6136ddc2f1087c9 in term 0.
I20260810 18:56:12.292416 20741 leader_election.cc:302] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2954a80f7f244c6fa724e0998da9017c, 5de668daa1214b7fb6136ddc2f1087c9; no voters:
I20260810 18:56:12.292742 20941 raft_consensus.cc:2781] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:56:12.292820 20941 raft_consensus.cc:482] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:56:12.292886 20941 raft_consensus.cc:3037] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:12.293927 20941 raft_consensus.cc:504] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.294276 20941 leader_election.cc:288] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [CANDIDATE]: Term 1 election: Requested vote from peers 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669), d94f3a58760c4c99b09c2cce7d283f83 (127.30.153.129:35839)
I20260810 18:56:12.294688 20646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8866182f8414adcbe54291e219b25b4" candidate_uuid: "5de668daa1214b7fb6136ddc2f1087c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2954a80f7f244c6fa724e0998da9017c"
I20260810 18:56:12.294838 20435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8866182f8414adcbe54291e219b25b4" candidate_uuid: "5de668daa1214b7fb6136ddc2f1087c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d94f3a58760c4c99b09c2cce7d283f83"
I20260810 18:56:12.294844 20646 raft_consensus.cc:3037] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:12.294953 20435 raft_consensus.cc:3037] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:12.295948 20435 raft_consensus.cc:2454] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5de668daa1214b7fb6136ddc2f1087c9 in term 1.
I20260810 18:56:12.296068 20646 raft_consensus.cc:2454] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5de668daa1214b7fb6136ddc2f1087c9 in term 1.
I20260810 18:56:12.296363 20741 leader_election.cc:302] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5de668daa1214b7fb6136ddc2f1087c9, d94f3a58760c4c99b09c2cce7d283f83; no voters:
I20260810 18:56:12.296659 20941 raft_consensus.cc:2781] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:12.296938 20941 raft_consensus.cc:686] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 LEADER]: Becoming Leader. State: Replica: 5de668daa1214b7fb6136ddc2f1087c9, State: Running, Role: LEADER
I20260810 18:56:12.297290 20941 consensus_queue.cc:237] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.300487 20216 catalog_manager.cc:5495] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131). New cstate: current_term: 1 leader_uuid: "5de668daa1214b7fb6136ddc2f1087c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } health_report { overall_health: UNKNOWN } } }
I20260810 18:56:12.343173 20925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.339582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38206 (local address 127.30.153.131:35529)
0810 18:56:12.339739 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:12.339743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:12.339757 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:56:12.339839 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:12.339841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:12.339899 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:12.340003 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:56:12.340235 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.340774 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.341621 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:12.341849 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:12.341933 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:12.342301 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:56:12.342752 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:56:12.342851 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:12.342992 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 18:56:12.343046 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 18:56:12.349747 20435 raft_consensus.cc:1261] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Refusing update from remote peer 5de668daa1214b7fb6136ddc2f1087c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:12.349802 20646 raft_consensus.cc:1261] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 1 FOLLOWER]: Refusing update from remote peer 5de668daa1214b7fb6136ddc2f1087c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:12.350566 20946 consensus_queue.cc:1035] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:12.350731 20941 consensus_queue.cc:1035] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:56:12.351747 20913 tablet.cc:2369] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:56:12.358875 20955 mvcc.cc:205] Tried to move back new op lower bound from 7317045953937014784 to 7317045953732227072. Current Snapshot: MvccSnapshot[applied={T|T < 7317045953937014784}]
I20260810 18:56:12.360742 20957 mvcc.cc:205] Tried to move back new op lower bound from 7317045953937014784 to 7317045953732227072. Current Snapshot: MvccSnapshot[applied={T|T < 7317045953937014784}]
I20260810 18:56:12.367383 20216 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:12.367583 20216 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:12.370074 20216 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Sending DeleteTablet for 3 replicas of tablet c8866182f8414adcbe54291e219b25b4
I20260810 18:56:12.370766 20808 tablet_service.cc:1514] Processing DeleteTablet for tablet c8866182f8414adcbe54291e219b25b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:12 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:38194
I20260810 18:56:12.370883 20617 tablet_service.cc:1514] Processing DeleteTablet for tablet c8866182f8414adcbe54291e219b25b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:12 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:55114
I20260810 18:56:12.371088 20404 tablet_service.cc:1514] Processing DeleteTablet for tablet c8866182f8414adcbe54291e219b25b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:12 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:54762
I20260810 18:56:12.371280 20964 tablet_replica.cc:330] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: stopping tablet replica
I20260810 18:56:12.371485 20965 tablet_replica.cc:330] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: stopping tablet replica
I20260810 18:56:12.371609 20964 raft_consensus.cc:2229] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:56:12.371721 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 20279
I20260810 18:56:12.371938 20965 raft_consensus.cc:2229] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:56:12.372030 20964 raft_consensus.cc:2258] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:12.377255 20964 ts_tablet_manager.cc:1905] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:12.380442 20964 ts_tablet_manager.cc:1918] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:12.380534 20964 log.cc:1196] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/c8866182f8414adcbe54291e219b25b4
I20260810 18:56:12.380852 20964 ts_tablet_manager.cc:1939] T c8866182f8414adcbe54291e219b25b4 P 5de668daa1214b7fb6136ddc2f1087c9: Deleting consensus metadata
I20260810 18:56:12.382000 20194 catalog_manager.cc:4841] TS 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529): tablet c8866182f8414adcbe54291e219b25b4 (table test-table [id=4a3a32c28c5e4bdcbe864efb629711ec]) successfully deleted
I20260810 18:56:12.384210 20965 raft_consensus.cc:2258] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:12.385689 20965 ts_tablet_manager.cc:1905] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:12.389071 20965 ts_tablet_manager.cc:1918] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:12.389189 20965 log.cc:1196] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/c8866182f8414adcbe54291e219b25b4
I20260810 18:56:12.389544 20965 ts_tablet_manager.cc:1939] T c8866182f8414adcbe54291e219b25b4 P 2954a80f7f244c6fa724e0998da9017c: Deleting consensus metadata
I20260810 18:56:12.390930 20191 catalog_manager.cc:4841] TS 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669): tablet c8866182f8414adcbe54291e219b25b4 (table test-table [id=4a3a32c28c5e4bdcbe864efb629711ec]) successfully deleted
W20260810 18:56:12.394073 20191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:35839 (error 108)
I20260810 18:56:12.395212 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.394893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:35839 (local address 127.0.0.1:54764)
0810 18:56:12.395017 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.395094 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:35839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 18:56:12.395387 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 20506
W20260810 18:56:12.395428 20191 catalog_manager.cc:4584] TS d94f3a58760c4c99b09c2cce7d283f83 (127.30.153.129:35839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8866182f8414adcbe54291e219b25b4: Network error: Client connection negotiation failed: client connection to 127.30.153.129:35839: connect: Connection refused (error 111)
I20260810 18:56:12.408499 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 20721
I20260810 18:56:12.422256 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 20166
I20260810 18:56:12.438815 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38663
--webserver_interface=127.30.153.190
--webserver_port=39095
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:38663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:12.550176 20969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:12.550467 20969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:12.550531 20969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:12.554633 20969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:12.554690 20969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:12.554713 20969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:12.554728 20969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:12.554744 20969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:12.559957 20969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:38663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=39095
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:12.560987 20969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:12.562000 20969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:56:12.570843 20969 server_base.cc:1034] running on GCE node
W20260810 18:56:12.570763 20980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:12.570791 20982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:12.570760 20979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:12.571820 20969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:12.572913 20969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:12.574118 20969 hybrid_clock.cc:648] HybridClock initialized: now 1786388172574099 us; error 43 us; skew 500 ppm
Aug 10 18:56:12 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388172, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:12.577039 20969 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:56:12.577850 20969 webserver.cc:460] Webserver started at http://127.30.153.190:39095/ using document root <none> and password file <none>
I20260810 18:56:12.578363 20969 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:12.578486 20969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:12.582250 20969 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:56:12.584388 20991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:12.585707 20969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:56:12.585812 20969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "54f600d408ad4aa2a38ced4bb38749b0"
format_stamp: "Formatted at 2026-08-10 18:56:09 on dist-test-slave-68xk"
server_key: "dd94ad69273798035910d33955c9db5c"
server_key_iv: "6caee141edbf2a8a2f7263021d9c6c84"
server_key_version: "encryptionkey@0"
I20260810 18:56:12.586329 20969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:12.606135 20969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:12.609241 20969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:12.609400 20969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:12.616672 20969 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:38663
I20260810 18:56:12.616986 21072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:38663 every 8 connection(s)
I20260810 18:56:12.617801 20969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:56:12.622128 21074 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:56:12.623832 21074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Bootstrap starting.
I20260810 18:56:12.626245 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 20969
I20260810 18:56:12.626717 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:35839
--local_ip_for_outbound_sockets=127.30.153.129
--tserver_master_addrs=127.30.153.190:38663
--webserver_port=37481
--webserver_interface=127.30.153.129
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:12.642194 21074 log.cc:824] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:12.654004 21074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:56:12.654549 21074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0: Bootstrap complete.
I20260810 18:56:12.659269 21074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:12.659723 21074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f600d408ad4aa2a38ced4bb38749b0, State: Initialized, Role: FOLLOWER
I20260810 18:56:12.660285 21074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:12.660454 21074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:56:12.660557 21074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:56:12.660696 21074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:56:12.662966 21074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:12.663358 21074 leader_election.cc:302] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54f600d408ad4aa2a38ced4bb38749b0; no voters:
I20260810 18:56:12.663794 21074 leader_election.cc:288] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 18:56:12.663897 21081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:56:12.664143 21081 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [term 3 LEADER]: Becoming Leader. State: Replica: 54f600d408ad4aa2a38ced4bb38749b0, State: Running, Role: LEADER
I20260810 18:56:12.664469 21081 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } }
I20260810 18:56:12.665222 21074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:12.665531 21081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54f600d408ad4aa2a38ced4bb38749b0. Latest consensus state: current_term: 3 leader_uuid: "54f600d408ad4aa2a38ced4bb38749b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } } }
I20260810 18:56:12.665675 21081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:12.666101 21087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:56:12.666111 21082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "54f600d408ad4aa2a38ced4bb38749b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f600d408ad4aa2a38ced4bb38749b0" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 38663 } } }
I20260810 18:56:12.666373 21082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f600d408ad4aa2a38ced4bb38749b0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:12.672184 21087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:56:12.673110 21087 catalog_manager.cc:1259] Loaded cluster ID: 6e07aac6588d4b65a8a5b22475737136
I20260810 18:56:12.673209 21087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 18:56:12.674214 21087 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd457a6f9ad google::LogMessage::Fail() at ??:0
@ 0x7fd457a7288a google::LogMessage::SendToLog() at ??:0
@ 0x7fd457a6f4de google::LogMessage::Flush() at ??:0
@ 0x7fd457a70a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd4623aefdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fd4623afd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd4623ac579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fd4623f0640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd459cc13d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd459cc1c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fd459cc33f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fd459cb2512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd45b2f66db start_thread at ??:0
@ 0x7fd45656d61f clone at ??:0
W20260810 18:56:12.775777 21077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:12.776034 21077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:12.776081 21077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:12.780122 21077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:12.780184 21077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:12.780243 21077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:56:12.785672 21077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:35839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=37481
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:12.786859 21077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:12.787931 21077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:12.798098 21138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:12.798131 21077 server_base.cc:1034] running on GCE node
W20260810 18:56:12.798357 21139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:12.798401 21141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:12.798997 21077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:12.799841 21077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:12.801057 21077 hybrid_clock.cc:648] HybridClock initialized: now 1786388172801024 us; error 44 us; skew 500 ppm
Aug 10 18:56:12 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388172, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:12.809433 21077 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:56:12.810500 21077 webserver.cc:460] Webserver started at http://127.30.153.129:37481/ using document root <none> and password file <none>
I20260810 18:56:12.811108 21077 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:12.811175 21077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:12.815341 21077 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:56:12.817677 21154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:12.819080 21077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:56:12.819227 21077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "d94f3a58760c4c99b09c2cce7d283f83"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "d16afb6d674cd99eba6bffa668cc3e6f"
server_key_iv: "2414767960c999886bdf016e405216d5"
server_key_version: "encryptionkey@0"
I20260810 18:56:12.819782 21077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:12.843582 21077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:12.847236 21077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:12.847460 21077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:12.848120 21077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:12.849347 21167 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 18:56:12.855643 21077 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:56:12.855767 21077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.000s
I20260810 18:56:12.855893 21077 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:56:12.857793 21077 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:56:12.857877 21077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 18:56:12.858178 21167 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Bootstrap starting.
I20260810 18:56:12.870355 21077 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:35839
I20260810 18:56:12.870702 21345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:35839 every 8 connection(s)
I20260810 18:56:12.871569 21077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:56:12.875332 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 21077
I20260810 18:56:12.875898 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:39669
--local_ip_for_outbound_sockets=127.30.153.130
--tserver_master_addrs=127.30.153.190:38663
--webserver_port=33175
--webserver_interface=127.30.153.130
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:12.882725 21167 log.cc:824] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:12.893642 21167 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:56:12.894153 21167 tablet_bootstrap.cc:492] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Bootstrap complete.
I20260810 18:56:12.895838 21167 ts_tablet_manager.cc:1397] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Time spent bootstrapping tablet: real 0.038s user 0.029s sys 0.007s
W20260810 18:56:12.896569 21349 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:56:12.873816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:59859)
0810 18:56:12.874390 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.874480 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:12.875786 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:12.895450 (+ 19664us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":172,"threads_started":1}
W20260810 18:56:12.897209 21346 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:38663 (0 consecutive failures): Network error: Failed to ping master at 127.30.153.190:38663: Client connection negotiation failed: client connection to 127.30.153.190:38663: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:56:12.898046 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.897746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:45343)
0810 18:56:12.897897 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.897975 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 18:56:12.898898 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.898646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:34547)
0810 18:56:12.898779 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.898833 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:56:12.899185 21346 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:56:12.899350 21167 raft_consensus.cc:348] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.899823 21167 raft_consensus.cc:729] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d94f3a58760c4c99b09c2cce7d283f83, State: Initialized, Role: FOLLOWER
I20260810 18:56:12.900300 21167 consensus_queue.cc:260] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:12.901424 21167 ts_tablet_manager.cc:1428] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260810 18:56:12.901429 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.901190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:54387)
0810 18:56:12.901311 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.901367 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 18:56:12.902216 21349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:12.902003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:39029)
0810 18:56:12.902109 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:12.902166 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 18:56:13.005846 21350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:13.006188 21350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:13.006250 21350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:13.011823 21350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:13.011898 21350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:13.011981 21350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:56:13.019058 21350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:39669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=33175
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:13.020248 21350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:13.021370 21350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:13.031791 21360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:13.031783 21365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:13.032122 21350 server_base.cc:1034] running on GCE node
W20260810 18:56:13.031778 21367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:13.032770 21350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:13.033584 21350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:13.034798 21350 hybrid_clock.cc:648] HybridClock initialized: now 1786388173034726 us; error 83 us; skew 500 ppm
Aug 10 18:56:13 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:13.038807 21350 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:56:13.043676 21350 webserver.cc:460] Webserver started at http://127.30.153.130:33175/ using document root <none> and password file <none>
I20260810 18:56:13.044508 21350 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:13.044596 21350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:13.050556 21350 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 18:56:13.053517 21376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:13.055152 21350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:56:13.055344 21350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "2954a80f7f244c6fa724e0998da9017c"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "ba37f922e3aa6a9fadd04c4d74d6affe"
server_key_iv: "2eb35a0bdc6969cefdb6f194dcc14b04"
server_key_version: "encryptionkey@0"
I20260810 18:56:13.056176 21350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:13.073058 21350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:13.077142 21350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:13.077363 21350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:13.078015 21350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:13.079008 21350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:13.079082 21350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:13.079152 21350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:13.079216 21350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:13.090716 21350 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:39669
I20260810 18:56:13.090736 21561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:39669 every 8 connection(s)
I20260810 18:56:13.091791 21350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:56:13.094595 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 21350
I20260810 18:56:13.095093 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:35529
--local_ip_for_outbound_sockets=127.30.153.131
--tserver_master_addrs=127.30.153.190:38663
--webserver_port=42851
--webserver_interface=127.30.153.131
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:13.096115 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:13.094060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:49433)
0810 18:56:13.094728 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:13.095242 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":184,"threads_started":1}
W20260810 18:56:13.096653 21563 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:38663 (0 consecutive failures): Network error: Failed to ping master at 127.30.153.190:38663: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
I20260810 18:56:13.097477 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:13.097182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:54115)
0810 18:56:13.097326 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:13.097396 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 18:56:13.098172 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:13.097912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:33843)
0810 18:56:13.098030 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:13.098092 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 18:56:13.098403 21563 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:56:13.209697 21566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:13.209968 21566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:13.210012 21566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:13.214051 21566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:13.214107 21566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:13.214164 21566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:56:13.219362 21566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:35529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=42851
--tserver_master_addrs=127.30.153.190:38663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:13.220360 21566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:13.221344 21566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:13.231467 21577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:13.231473 21578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:13.231473 21580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:13.231665 21566 server_base.cc:1034] running on GCE node
I20260810 18:56:13.232142 21566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:13.232782 21566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:13.233978 21566 hybrid_clock.cc:648] HybridClock initialized: now 1786388173233969 us; error 31 us; skew 500 ppm
Aug 10 18:56:13 dist-test-slave-68xk krb5kdc[19346](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:13.236842 21566 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:56:13.237746 21566 webserver.cc:460] Webserver started at http://127.30.153.131:42851/ using document root <none> and password file <none>
I20260810 18:56:13.238336 21566 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:13.238457 21566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:13.242709 21566 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260810 18:56:13.244969 21587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:13.246197 21566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:56:13.246368 21566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "5de668daa1214b7fb6136ddc2f1087c9"
format_stamp: "Formatted at 2026-08-10 18:56:10 on dist-test-slave-68xk"
server_key: "85f5977edcb9d38df7df01134e1a425f"
server_key_iv: "243c1c050face4b5bc6373137ae07f0e"
server_key_version: "encryptionkey@0"
I20260810 18:56:13.247012 21566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:13.273371 21566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:13.276597 21566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:13.276825 21566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:13.277470 21566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:13.278355 21566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:13.278457 21566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:13.278551 21566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:13.278605 21566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:13.288101 21566 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:35529
I20260810 18:56:13.288129 21761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:35529 every 8 connection(s)
I20260810 18:56:13.289165 21566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:56:13.293097 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:13.291431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:41603)
0810 18:56:13.291895 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:13.292359 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":107,"threads_started":1}
I20260810 18:56:13.293401 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 21566
W20260810 18:56:13.293634 21762 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:38663 (0 consecutive failures): Network error: Failed to ping master at 127.30.153.190:38663: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
I20260810 18:56:13.294349 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:13.294066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:58113)
0810 18:56:13.294181 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:13.294270 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 18:56:13.295089 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:13.294878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:39169)
0810 18:56:13.294985 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:13.295031 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 18:56:13.295295 21762 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:56:13.903584 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:13.903071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:35937)
0810 18:56:13.903391 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:13.903460 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":82,"threads_started":1}
I20260810 18:56:14.099347 21769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:14.098957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:44695)
0810 18:56:14.099185 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:14.099247 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
I20260810 18:56:14.138684 21771 raft_consensus.cc:482] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:14.138952 21771 raft_consensus.cc:504] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:14.140132 21771 leader_election.cc:288] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529), 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669)
Aug 10 18:56:14 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388172, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.130@KRBTEST.COM
Aug 10 18:56:14 dist-test-slave-68xk krb5kdc[19346](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388172, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.131@KRBTEST.COM
I20260810 18:56:14.161739 21774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:14.140687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:58643 (local address 127.30.153.131:35529)
0810 18:56:14.141065 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:14.141083 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:14.141130 (+ 47us) server_negotiation.cc:407] Connection header received
0810 18:56:14.142005 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:14.142014 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:14.142507 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:14.142850 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:14.144673 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:14.146261 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:14.148280 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:14.149009 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:14.149185 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:14.156615 (+ 7430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:14.156620 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:14.156630 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:14.156662 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:14.158724 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:14.159403 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:14.159408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:14.159412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:14.159531 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:14.159931 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:14.159934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:14.159935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:14.160419 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:14.160561 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:14.161225 (+ 664us) server_negotiation.cc:299] Negotiation successful
0810 18:56:14.161540 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":88,"threads_started":1}
I20260810 18:56:14.161813 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:14.140438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35529 (local address 127.30.153.129:58643)
0810 18:56:14.140643 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:14.140661 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:14.140793 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:14.142991 (+ 2198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:14.142997 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:14.143641 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:14.144461 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:14.144484 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:14.146463 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:14.146467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:14.148115 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:14.148128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:14.149171 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:14.149175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:14.149368 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:14.153948 (+ 4580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:14.153978 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:14.156384 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:14.158876 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:14.158880 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:14.158882 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:14.159276 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:14.159680 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:14.159682 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:14.159684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:14.159794 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:14.160561 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:14.160568 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:14.160876 (+ 308us) client_negotiation.cc:769] Sending connection context
0810 18:56:14.161118 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 18:56:14.161618 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":462,"spinlock_wait_cycles":9984}
I20260810 18:56:14.162029 21773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:14.140591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:54149 (local address 127.30.153.130:39669)
0810 18:56:14.141065 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:14.141089 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:14.141130 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:56:14.142025 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:14.142032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:14.142507 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:14.142850 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:14.144673 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:14.146261 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:14.148271 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:14.149009 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:14.149185 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:14.156332 (+ 7147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:14.156340 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:14.156361 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:14.156401 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:14.158708 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:14.159332 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:14.159342 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:14.159350 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:14.159504 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:14.159898 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:14.159902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:14.159903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:14.160582 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:14.160757 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:14.161520 (+ 763us) server_negotiation.cc:299] Negotiation successful
0810 18:56:14.161809 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":118,"threads_started":1}
I20260810 18:56:14.161811 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:14.140438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:39669 (local address 127.30.153.129:54149)
0810 18:56:14.140921 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:14.140936 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:14.141051 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:14.142991 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:14.142997 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:14.143641 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:14.144462 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:14.144484 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:14.146463 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:14.146467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:14.148115 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:14.148128 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:14.149722 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:14.149728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:14.149843 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:14.153948 (+ 4105us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:14.153972 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:14.155943 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:14.158846 (+ 2903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:14.158854 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:14.158871 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:14.159197 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:14.159623 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:14.159629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:14.159634 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:14.159786 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:14.160811 (+ 1025us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:14.160815 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:14.160892 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 18:56:14.161118 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 18:56:14.161555 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":428,"spinlock_wait_cycles":1920,"thread_start_us":82,"threads_started":1}
W20260810 18:56:14.164746 21159 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
W20260810 18:56:14.164929 21156 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
I20260810 18:56:14.165051 21156 leader_election.cc:302] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d94f3a58760c4c99b09c2cce7d283f83; no voters: 2954a80f7f244c6fa724e0998da9017c, 5de668daa1214b7fb6136ddc2f1087c9
I20260810 18:56:14.165413 21771 raft_consensus.cc:2726] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:56:14.296344 21775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:14.295913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:47225)
0810 18:56:14.296174 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:14.296228 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":88,"threads_started":1}
I20260810 18:56:14.904955 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:14.904482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:54357)
0810 18:56:14.904791 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:14.904853 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":85,"threads_started":1}
I20260810 18:56:15.100525 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:15.100097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:35151)
0810 18:56:15.100375 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:15.100433 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":84,"threads_started":1}
I20260810 18:56:15.299543 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:15.299098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:52999)
0810 18:56:15.299375 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:15.299439 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":101,"threads_started":1}
I20260810 18:56:15.906353 21782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:15.905811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:37219)
0810 18:56:15.906115 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:15.906209 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":79,"threads_started":1}
I20260810 18:56:16.099607 21783 raft_consensus.cc:482] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:16.099747 21783 raft_consensus.cc:504] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:16.100297 21783 leader_election.cc:288] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529), 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669)
W20260810 18:56:16.101106 21159 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
W20260810 18:56:16.101238 21156 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
I20260810 18:56:16.101312 21156 leader_election.cc:302] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d94f3a58760c4c99b09c2cce7d283f83; no voters: 2954a80f7f244c6fa724e0998da9017c, 5de668daa1214b7fb6136ddc2f1087c9
I20260810 18:56:16.101475 21783 raft_consensus.cc:2726] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:56:16.101851 21784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:16.101391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:41659)
0810 18:56:16.101685 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:16.101753 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":98,"threads_started":1}
I20260810 18:56:16.300855 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:16.300388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:48613)
0810 18:56:16.300693 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:16.300755 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":101,"threads_started":1}
I20260810 18:56:16.907624 21790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:16.907135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:43329)
0810 18:56:16.907441 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:16.907503 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":102,"threads_started":1}
I20260810 18:56:17.103188 21796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:17.102704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:48039)
0810 18:56:17.103011 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:17.103084 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":84,"threads_started":1}
I20260810 18:56:17.302160 21799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:17.301646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:54641)
0810 18:56:17.301957 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:17.302018 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":94,"threads_started":1}
I20260810 18:56:17.909080 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:17.908580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:34695)
0810 18:56:17.908913 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:17.908978 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
I20260810 18:56:18.104440 21804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:18.103959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:53279)
0810 18:56:18.104285 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:18.104347 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":82,"threads_started":1}
I20260810 18:56:18.303498 21805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:18.302967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:57467)
0810 18:56:18.303325 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:18.303393 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
I20260810 18:56:18.910365 21806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:18.909913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:55713)
0810 18:56:18.910202 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:18.910262 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":104,"threads_started":1}
I20260810 18:56:19.089030 21808 raft_consensus.cc:482] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:19.089216 21808 raft_consensus.cc:504] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:19.089710 21808 leader_election.cc:288] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529), 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669)
W20260810 18:56:19.090610 21159 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
W20260810 18:56:19.090752 21156 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
I20260810 18:56:19.090853 21156 leader_election.cc:302] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d94f3a58760c4c99b09c2cce7d283f83; no voters: 2954a80f7f244c6fa724e0998da9017c, 5de668daa1214b7fb6136ddc2f1087c9
I20260810 18:56:19.091078 21808 raft_consensus.cc:2726] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:56:19.105595 21809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:19.105176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:45735)
0810 18:56:19.105450 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:19.105507 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":93,"threads_started":1}
I20260810 18:56:19.304674 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:19.304262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:58243)
0810 18:56:19.304520 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:19.304577 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":79,"threads_started":1}
I20260810 18:56:19.911753 21811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:19.911271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:57851)
0810 18:56:19.911556 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:19.911646 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":81,"threads_started":1}
I20260810 18:56:20.107045 21812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:20.106522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:60681)
0810 18:56:20.106885 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:20.106946 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":78,"threads_started":1}
I20260810 18:56:20.306283 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:20.305609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:47063)
0810 18:56:20.306032 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:20.306133 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":125,"threads_started":1}
I20260810 18:56:20.913139 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:20.912669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:52779)
0810 18:56:20.912972 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:20.913029 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":104,"threads_started":1}
I20260810 18:56:21.108287 21815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:21.107837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:54605)
0810 18:56:21.108127 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:21.108184 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":123,"threads_started":1}
I20260810 18:56:21.307533 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:21.307088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:44455)
0810 18:56:21.307380 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:21.307438 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":78,"threads_started":1}
I20260810 18:56:21.914642 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:21.914066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:41159)
0810 18:56:21.914450 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:21.914525 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":143,"threads_started":1}
I20260810 18:56:22.109665 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:22.109090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:44537)
0810 18:56:22.109442 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:22.109528 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":107,"threads_started":1}
I20260810 18:56:22.308807 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:22.308371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:52549)
0810 18:56:22.308647 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:22.308709 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
I20260810 18:56:22.916071 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:22.915600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:34051)
0810 18:56:22.915907 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:22.915966 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":120,"threads_started":1}
I20260810 18:56:22.992628 21825 raft_consensus.cc:482] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:22.992782 21825 raft_consensus.cc:504] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:22.993304 21825 leader_election.cc:288] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529), 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669)
W20260810 18:56:22.994350 21159 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
W20260810 18:56:22.994541 21156 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
I20260810 18:56:22.994629 21156 leader_election.cc:302] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d94f3a58760c4c99b09c2cce7d283f83; no voters: 2954a80f7f244c6fa724e0998da9017c, 5de668daa1214b7fb6136ddc2f1087c9
I20260810 18:56:22.994839 21825 raft_consensus.cc:2726] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:56:23.111202 21826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:23.110595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:44983)
0810 18:56:23.110953 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:23.111042 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":104,"threads_started":1}
I20260810 18:56:23.310302 21828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:23.309785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:39909)
0810 18:56:23.310118 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:23.310191 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":92,"threads_started":1}
I20260810 18:56:23.917435 21829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:23.916919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:49575)
0810 18:56:23.917249 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:23.917310 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":85,"threads_started":1}
I20260810 18:56:24.112771 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:24.112225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:48633)
0810 18:56:24.112577 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:24.112653 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":141,"threads_started":1}
I20260810 18:56:24.311532 21834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:24.311092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:45525)
0810 18:56:24.311366 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:24.311427 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
I20260810 18:56:24.918938 21835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:24.918374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:52281)
0810 18:56:24.918756 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:24.918821 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":104,"threads_started":1}
I20260810 18:56:25.114063 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:25.113589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:44339)
0810 18:56:25.113895 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:25.113962 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":86,"threads_started":1}
I20260810 18:56:25.312969 21837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:25.312405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:42791)
0810 18:56:25.312766 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:25.312843 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":98,"threads_started":1}
I20260810 18:56:25.321017 21838 raft_consensus.cc:482] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:25.321270 21838 raft_consensus.cc:504] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5de668daa1214b7fb6136ddc2f1087c9" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35529 } } peers { permanent_uuid: "2954a80f7f244c6fa724e0998da9017c" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 39669 } } peers { permanent_uuid: "d94f3a58760c4c99b09c2cce7d283f83" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 35839 } }
I20260810 18:56:25.322001 21838 leader_election.cc:288] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529), 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669)
W20260810 18:56:25.324499 21156 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2954a80f7f244c6fa724e0998da9017c (127.30.153.130:39669): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
W20260810 18:56:25.324962 21159 leader_election.cc:341] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5de668daa1214b7fb6136ddc2f1087c9 (127.30.153.131:35529): Not found: Tablet not found: c8866182f8414adcbe54291e219b25b4
I20260810 18:56:25.325070 21159 leader_election.cc:302] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d94f3a58760c4c99b09c2cce7d283f83; no voters: 2954a80f7f244c6fa724e0998da9017c, 5de668daa1214b7fb6136ddc2f1087c9
I20260810 18:56:25.325234 21838 raft_consensus.cc:2726] T c8866182f8414adcbe54291e219b25b4 P d94f3a58760c4c99b09c2cce7d283f83 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:56:25.920248 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:25.919786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:55971)
0810 18:56:25.920070 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:25.920135 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
I20260810 18:56:26.115387 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:26.114941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:42081)
0810 18:56:26.115231 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:26.115290 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
I20260810 18:56:26.314400 21841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:26.313885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:53173)
0810 18:56:26.314236 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:26.314302 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":102,"threads_started":1}
I20260810 18:56:26.921538 21845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:26.921113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:45627)
0810 18:56:26.921380 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:26.921438 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260810 18:56:27.116931 21846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:27.116465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:59617)
0810 18:56:27.116776 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:27.116831 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":121,"threads_started":1}
I20260810 18:56:27.315645 21847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:27.315228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:43821)
0810 18:56:27.315491 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:27.315547 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":96,"threads_started":1}
I20260810 18:56:27.923319 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:27.922750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.129:43131)
0810 18:56:27.923065 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:27.923176 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":98,"threads_started":1}
I20260810 18:56:28.118356 21852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:28.117919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.130:47219)
0810 18:56:28.118199 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:28.118257 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
I20260810 18:56:28.296484 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 21077
I20260810 18:56:28.315912 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 21350
I20260810 18:56:28.322089 21853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:28.320713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:38663 (local address 127.30.153.131:49309)
0810 18:56:28.321910 (+ 1197us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:28.321983 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:38663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1085,"thread_start_us":968,"threads_started":1}
I20260810 18:56:28.328069 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 21566
2026-08-10T18:56:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19900 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:56:28 dist-test-slave-68xk krb5kdc[21859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:56:28 dist-test-slave-68xk krb5kdc[21859](info): set up 2 sockets
Aug 10 18:56:28 dist-test-slave-68xk krb5kdc[21859](info): commencing operation
krb5kdc: starting...
W20260810 18:56:29.671875 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:56:29 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:56:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:56:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:56:29.805177 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:37857
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:37857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:29.923854 21877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:29.924117 21877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:29.924165 21877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:29.928254 21877 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:56:29.928308 21877 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:56:29.928323 21877 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:56:29.928339 21877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:29.928356 21877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:29.928414 21877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:29.928442 21877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:29.928476 21877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:29.933933 21877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:37857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:37857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:29.935103 21877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:29.936214 21877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:29.945575 21888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:29.945533 21893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:29.945580 21877 server_base.cc:1034] running on GCE node
W20260810 18:56:29.945531 21895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:29.946542 21877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:29.947584 21877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:29.948756 21877 hybrid_clock.cc:648] HybridClock initialized: now 1786388189948728 us; error 42 us; skew 500 ppm
Aug 10 18:56:29 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:29.951776 21877 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:56:29.952687 21877 webserver.cc:460] Webserver started at http://127.30.153.190:43891/ using document root <none> and password file <none>
I20260810 18:56:29.953284 21877 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:29.953385 21877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:29.953624 21877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:29.955760 21877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "607d12b05f9c4034b63f2dd8b072ec4a"
format_stamp: "Formatted at 2026-08-10 18:56:29 on dist-test-slave-68xk"
server_key: "a0d427b1605b507c88bfe0df527ed5f4"
server_key_iv: "6038b2a9ddc2ce6d7ba0426edc4b314d"
server_key_version: "encryptionkey@0"
I20260810 18:56:29.956344 21877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "607d12b05f9c4034b63f2dd8b072ec4a"
format_stamp: "Formatted at 2026-08-10 18:56:29 on dist-test-slave-68xk"
server_key: "a0d427b1605b507c88bfe0df527ed5f4"
server_key_iv: "6038b2a9ddc2ce6d7ba0426edc4b314d"
server_key_version: "encryptionkey@0"
I20260810 18:56:29.960178 21877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:56:29.962700 21906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:29.963937 21877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:56:29.964078 21877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "607d12b05f9c4034b63f2dd8b072ec4a"
format_stamp: "Formatted at 2026-08-10 18:56:29 on dist-test-slave-68xk"
server_key: "a0d427b1605b507c88bfe0df527ed5f4"
server_key_iv: "6038b2a9ddc2ce6d7ba0426edc4b314d"
server_key_version: "encryptionkey@0"
I20260810 18:56:29.964188 21877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:29.979097 21877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:29.982069 21877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:29.982278 21877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:29.989581 21877 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:37857
I20260810 18:56:29.989903 21985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:37857 every 8 connection(s)
I20260810 18:56:29.990613 21877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:56:29.992781 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 21877
I20260810 18:56:29.992929 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:56:29.993230 31334 external_mini_cluster.cc:1428] Setting key 8afe0d9b4a717a56a295caf57854ffde
I20260810 18:56:29.993798 21986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:30.001698 21986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Bootstrap starting.
Aug 10 18:56:30 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:30.004667 21986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:30.005812 21986 log.cc:824] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:30.008517 21986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: No bootstrap required, opened a new log
I20260810 18:56:30.010391 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:29.994742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54302 (local address 127.30.153.190:37857)
0810 18:56:29.995185 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:29.995206 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:29.995255 (+ 49us) server_negotiation.cc:407] Connection header received
0810 18:56:29.996200 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:29.996225 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:29.996683 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:29.997087 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:29.997924 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:29.999654 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.000561 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.001310 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.001497 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.004539 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:30.004550 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:30.004573 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:30.004617 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:30.006998 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.007493 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.007500 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.007507 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.007675 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.007954 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.007957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.007959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.008516 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:30.008743 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:30.009035 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 18:56:30.009371 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":113,"threads_started":1}
I20260810 18:56:30.013458 21986 raft_consensus.cc:348] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:56:30.013715 21986 raft_consensus.cc:374] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:30.013765 21986 raft_consensus.cc:729] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607d12b05f9c4034b63f2dd8b072ec4a, State: Initialized, Role: FOLLOWER
I20260810 18:56:30.014218 21986 consensus_queue.cc:260] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:56:30.014355 21986 raft_consensus.cc:388] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:56:30.014460 21986 raft_consensus.cc:482] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:56:30.014566 21986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:30.015550 21986 raft_consensus.cc:504] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:56:30.015877 21986 leader_election.cc:302] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 607d12b05f9c4034b63f2dd8b072ec4a; no voters:
I20260810 18:56:30.016182 21986 leader_election.cc:288] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:56:30.016479 21998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:30.016817 21998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 1 LEADER]: Becoming Leader. State: Replica: 607d12b05f9c4034b63f2dd8b072ec4a, State: Running, Role: LEADER
I20260810 18:56:30.017158 21998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:56:30.017606 21986 sys_catalog.cc:564] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:30.019333 22000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } } }
I20260810 18:56:30.019321 22001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 607d12b05f9c4034b63f2dd8b072ec4a. Latest consensus state: current_term: 1 leader_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } } }
I20260810 18:56:30.019491 22000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:30.019491 22001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:30.019976 22018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:56:30.023351 22018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:56:30.029130 22018 catalog_manager.cc:1347] Generated new cluster ID: 0d144b340b934498be8ff28af848ba40
I20260810 18:56:30.029202 22018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:56:30.057209 22018 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:56:30.058264 22018 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:56:30.065146 22018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Generated new TSK 0
I20260810 18:56:30.065899 22018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:56:30.102054 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37857
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:30.215866 22033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:30.216166 22033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:30.216241 22033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:30.220458 22033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:30.220541 22033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:30.220628 22033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:56:30.225732 22033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:30.226792 22033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:30.227798 22033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:30.237269 22040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:30.237316 22044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:30.237313 22041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:30.237452 22033 server_base.cc:1034] running on GCE node
I20260810 18:56:30.237808 22033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:30.238488 22033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:30.239655 22033 hybrid_clock.cc:648] HybridClock initialized: now 1786388190239639 us; error 38 us; skew 500 ppm
Aug 10 18:56:30 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:30.242592 22033 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:56:30.243436 22033 webserver.cc:460] Webserver started at http://127.30.153.129:43685/ using document root <none> and password file <none>
I20260810 18:56:30.244002 22033 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:30.244094 22033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:30.244364 22033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:30.246366 22033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "e78028d924414f698e690c07228ebd77"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "f874ed7e4518371edcdc19a5eeff1404"
server_key_iv: "520ad07a0cc240b0b9dad453573eabb0"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.246991 22033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "e78028d924414f698e690c07228ebd77"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "f874ed7e4518371edcdc19a5eeff1404"
server_key_iv: "520ad07a0cc240b0b9dad453573eabb0"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.250801 22033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:56:30.253127 22056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.254272 22033 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:56:30.254441 22033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "e78028d924414f698e690c07228ebd77"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "f874ed7e4518371edcdc19a5eeff1404"
server_key_iv: "520ad07a0cc240b0b9dad453573eabb0"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.254568 22033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:30.264134 22033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:30.267447 22033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:30.267643 22033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:30.268348 22033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:30.269325 22033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:30.269408 22033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.269495 22033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:30.269553 22033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.279470 22033 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:40599
I20260810 18:56:30.279582 22248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:40599 every 8 connection(s)
I20260810 18:56:30.280429 22033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:56:30.289896 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 22033
Aug 10 18:56:30 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:30.290073 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:56:30.290473 31334 external_mini_cluster.cc:1428] Setting key d25ec7546f321d34f6f6338fc4d53e2e
I20260810 18:56:30.295081 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.282323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:41253 (local address 127.30.153.190:37857)
0810 18:56:30.282516 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:30.282521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:30.283354 (+ 833us) server_negotiation.cc:407] Connection header received
0810 18:56:30.284637 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:30.284641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:30.284693 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:30.284782 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:30.285977 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.286531 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.287678 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.287907 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.288008 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.290768 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:30.290772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:30.290775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:30.290805 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:30.292516 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.293218 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.293221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.293222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.293298 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.293707 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.293710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.293711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.293889 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:30.293969 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:30.294709 (+ 740us) server_negotiation.cc:299] Negotiation successful
0810 18:56:30.294865 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 18:56:30.295706 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.282720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:41253)
0810 18:56:30.283200 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:30.283244 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:30.284402 (+ 1158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:30.284947 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:30.284954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:30.285358 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:30.285818 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.285832 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.286651 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.286654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.287562 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.287571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.288040 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.288043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.288213 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.288852 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:30.288868 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:30.290566 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:30.292641 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:30.292650 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:30.292689 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:30.293083 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:30.293393 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:30.293399 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:30.293403 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:30.293600 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:30.293998 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:30.294008 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:30.294375 (+ 367us) client_negotiation.cc:769] Sending connection context
0810 18:56:30.294595 (+ 220us) client_negotiation.cc:240] Negotiation successful
0810 18:56:30.294855 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":126,"threads_started":1}
I20260810 18:56:30.296937 22249 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37857
I20260810 18:56:30.297191 22249 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:30.297690 22249 heartbeater.cc:511] Master 127.30.153.190:37857 requested a full tablet report, sending...
I20260810 18:56:30.299268 21934 ts_manager.cc:194] Registered new tserver with Master: e78028d924414f698e690c07228ebd77 (127.30.153.129:40599)
I20260810 18:56:30.300413 21934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:41253
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:56:30.321849 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37857
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:30.441203 22255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:30.441435 22255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:30.441478 22255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:30.445500 22255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:30.445559 22255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:30.445616 22255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:56:30.450969 22255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:30.451997 22255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:30.453003 22255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:30.462731 22271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:30.462889 22267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:30.462788 22255 server_base.cc:1034] running on GCE node
W20260810 18:56:30.462750 22266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:30.463344 22255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:30.463974 22255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:30.465190 22255 hybrid_clock.cc:648] HybridClock initialized: now 1786388190465156 us; error 52 us; skew 500 ppm
Aug 10 18:56:30 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:30.468194 22255 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:56:30.469223 22255 webserver.cc:460] Webserver started at http://127.30.153.130:39011/ using document root <none> and password file <none>
I20260810 18:56:30.469827 22255 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:30.469930 22255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:30.470201 22255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:30.472421 22255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "521687247152450cb8ae19adb7434746"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "0229377f8d3e58d4985a9d546326c01f"
server_key_iv: "4aef83d67f4761fb83b0a44e2ecb30df"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.473042 22255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "521687247152450cb8ae19adb7434746"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "0229377f8d3e58d4985a9d546326c01f"
server_key_iv: "4aef83d67f4761fb83b0a44e2ecb30df"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.477432 22255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:56:30.480237 22281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.481556 22255 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:56:30.481711 22255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "521687247152450cb8ae19adb7434746"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "0229377f8d3e58d4985a9d546326c01f"
server_key_iv: "4aef83d67f4761fb83b0a44e2ecb30df"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.481843 22255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:30.500804 22255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:30.503870 22255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:30.504101 22255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:30.504731 22255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:30.505626 22255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:30.505703 22255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.505779 22255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:30.505831 22255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.515494 22255 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:34947
I20260810 18:56:30.515527 22468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:34947 every 8 connection(s)
I20260810 18:56:30.516431 22255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:56:30.519333 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 22255
I20260810 18:56:30.519483 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:56:30.519894 31334 external_mini_cluster.cc:1428] Setting key 28031d55a71472feb270b77e490cea35
Aug 10 18:56:30 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:56:30.531949 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.518471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:57783 (local address 127.30.153.190:37857)
0810 18:56:30.518606 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:30.518612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:30.519649 (+ 1037us) server_negotiation.cc:407] Connection header received
0810 18:56:30.521436 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:30.521438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:30.521481 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:30.521546 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:30.523223 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.523744 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.524924 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.525177 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.525297 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.528098 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:30.528101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:30.528103 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:30.528127 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:30.529636 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.530194 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.530198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.530199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.530276 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.530700 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.530703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.530704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.530912 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:30.531038 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:30.531647 (+ 609us) server_negotiation.cc:299] Negotiation successful
0810 18:56:30.531775 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 18:56:30.532838 22474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.518905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:57783)
0810 18:56:30.519447 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:30.519512 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:30.521185 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:30.521733 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:30.521742 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:30.522298 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:30.523058 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.523083 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.523889 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.523892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.524792 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.524801 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.525346 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.525350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.525521 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.526131 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:30.526154 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:30.527918 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:30.529776 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:30.529782 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:30.529797 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:30.530064 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:30.530375 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:30.530378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:30.530380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:30.530562 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:30.531036 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:30.531043 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:30.531366 (+ 323us) client_negotiation.cc:769] Sending connection context
0810 18:56:30.531576 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 18:56:30.531906 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":141,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
I20260810 18:56:30.534214 22469 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37857
I20260810 18:56:30.534559 22469 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:30.535270 22469 heartbeater.cc:511] Master 127.30.153.190:37857 requested a full tablet report, sending...
I20260810 18:56:30.536561 21934 ts_manager.cc:194] Registered new tserver with Master: 521687247152450cb8ae19adb7434746 (127.30.153.130:34947)
I20260810 18:56:30.537405 21934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:57783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:56:30.554028 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37857
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:30.682312 22478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:30.682657 22478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:30.682708 22478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:30.687297 22478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:30.687363 22478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:30.687429 22478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:56:30.693327 22478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:30.694464 22478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:30.695544 22478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:30.705794 22487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:30.705780 22478 server_base.cc:1034] running on GCE node
W20260810 18:56:30.705725 22488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:30.705816 22491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:30.706454 22478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:30.707113 22478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:30.708308 22478 hybrid_clock.cc:648] HybridClock initialized: now 1786388190708295 us; error 33 us; skew 500 ppm
Aug 10 18:56:30 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:30.711498 22478 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:56:30.712450 22478 webserver.cc:460] Webserver started at http://127.30.153.131:45651/ using document root <none> and password file <none>
I20260810 18:56:30.713084 22478 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:30.713194 22478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:30.713457 22478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:56:30.715816 22478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "8ef84dc10a6b414aba7292e25221221f"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "1473bfb3ebd8984a55c5cc81e7885560"
server_key_iv: "9ec100c3cd49fbb61aa415d52ccce7a2"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.716450 22478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "8ef84dc10a6b414aba7292e25221221f"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "1473bfb3ebd8984a55c5cc81e7885560"
server_key_iv: "9ec100c3cd49fbb61aa415d52ccce7a2"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.721141 22478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:56:30.723897 22502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.725312 22478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:56:30.725468 22478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "8ef84dc10a6b414aba7292e25221221f"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "1473bfb3ebd8984a55c5cc81e7885560"
server_key_iv: "9ec100c3cd49fbb61aa415d52ccce7a2"
server_key_version: "encryptionkey@0"
I20260810 18:56:30.725600 22478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:30.737488 22478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:30.740788 22478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:30.741015 22478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:30.741652 22478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:30.742592 22478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:30.742686 22478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.742775 22478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:30.742833 22478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:30.752080 22478 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:43551
I20260810 18:56:30.752092 22689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:43551 every 8 connection(s)
I20260810 18:56:30.752966 22478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:56:30.762441 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 22478
I20260810 18:56:30.762624 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:56:30.762980 31334 external_mini_cluster.cc:1428] Setting key 3e599599c1f2b2607fefe6abcda27f4a
Aug 10 18:56:30 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:30.767974 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.754979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:43265 (local address 127.30.153.190:37857)
0810 18:56:30.755119 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:30.755128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:30.756548 (+ 1420us) server_negotiation.cc:407] Connection header received
0810 18:56:30.757483 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:30.757487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:30.757545 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:30.757663 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:30.758999 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.759530 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.760701 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.760932 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.761047 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.763863 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:30.763871 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:30.763874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:30.763901 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:30.765617 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.766144 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.766147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.766148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.766222 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.766689 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.766696 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.766701 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.766902 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:30.766983 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:30.767748 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 18:56:30.767856 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:56:30.768977 22692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.755400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:43265)
0810 18:56:30.756393 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:30.756436 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:30.757255 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:30.757850 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:30.757857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:30.758287 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:30.758854 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.758868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.759664 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.759667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.760564 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.760573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.761088 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.761093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.761264 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.761821 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:30.761837 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:30.763687 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:30.765745 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:30.765750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:30.765763 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:30.766045 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:30.766313 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:30.766316 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:30.766317 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:30.766541 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:30.767051 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:30.767057 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:30.767441 (+ 384us) client_negotiation.cc:769] Sending connection context
0810 18:56:30.767683 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 18:56:30.767960 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":638,"threads_started":1}
I20260810 18:56:30.770069 22690 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37857
I20260810 18:56:30.770292 22690 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:30.770853 22690 heartbeater.cc:511] Master 127.30.153.190:37857 requested a full tablet report, sending...
I20260810 18:56:30.771844 21934 ts_manager.cc:194] Registered new tserver with Master: 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
I20260810 18:56:30.772451 21934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:43265
I20260810 18:56:30.777381 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:56:30.787431 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.779560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54308 (local address 127.30.153.190:37857)
0810 18:56:30.779817 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:30.779822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:30.779980 (+ 158us) server_negotiation.cc:407] Connection header received
0810 18:56:30.780079 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:30.780082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:30.780121 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:30.780206 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:30.780908 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.781421 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.782353 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.782618 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.782706 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.784067 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:30.784071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:30.784073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:30.784094 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:30.785550 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.786044 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.786046 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.786047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.786123 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:30.786474 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:30.786476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:30.786477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:30.786682 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:30.786788 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:30.787070 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 18:56:30.787209 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260810 18:56:30.790257 21934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:56:30.793376 21934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:56:30.809285 22705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.802797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:40599 (local address 127.0.0.1:49764)
0810 18:56:30.803202 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:30.803257 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:30.803376 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:30.804343 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:30.804349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:30.804371 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:30.804526 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.804536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.806074 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.806077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.807499 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.807511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.808682 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.808686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.808825 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.808869 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 18:56:30.808982 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 18:56:30.809105 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":11,"thread_start_us":105,"threads_started":1}
I20260810 18:56:30.809321 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.802937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49764 (local address 127.30.153.129:40599)
0810 18:56:30.803802 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:30.803809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:30.803835 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:56:30.803933 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:30.803938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:30.804170 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:30.804701 (+ 531us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:30.804843 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.805931 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.807628 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.808526 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.808698 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.808861 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:30.809037 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 18:56:30.809147 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":182,"threads_started":1}
I20260810 18:56:30.809594 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.803650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:34947 (local address 127.0.0.1:40984)
0810 18:56:30.803998 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:30.804015 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:30.804115 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:30.804853 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:30.804856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:30.804876 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:30.804959 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.804965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.806193 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.806197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.807645 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.807655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.808782 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.808786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.809177 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.809204 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:56:30.809287 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 18:56:30.809376 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":482,"thread_start_us":105,"threads_started":1}
I20260810 18:56:30.809629 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.803706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49772 (local address 127.30.153.131:43551)
0810 18:56:30.804380 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:30.804393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:30.804418 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:56:30.804499 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:30.804505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:30.804697 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:30.804866 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:30.805415 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.806567 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.808111 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.808707 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.808830 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.808956 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:30.809358 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 18:56:30.809465 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":168,"threads_started":1}
I20260810 18:56:30.809881 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.803022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:43551 (local address 127.0.0.1:49772)
0810 18:56:30.803412 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:30.803428 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:30.803539 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:30.805079 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:30.805082 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:30.805137 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:30.805267 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.805274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.806885 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.806888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.807994 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:30.808002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.809018 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.809021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:30.809112 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.809143 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:56:30.809214 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 18:56:30.809269 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":56,"thread_start_us":108,"threads_started":1}
I20260810 18:56:30.810122 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:30.803892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40984 (local address 127.30.153.130:34947)
0810 18:56:30.804380 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:30.804393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:30.804418 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:56:30.804499 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:30.804504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:30.804697 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:30.804866 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:30.805113 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.806041 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.807776 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:30.808637 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:30.808851 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:30.809006 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:30.809814 (+ 808us) server_negotiation.cc:299] Negotiation successful
0810 18:56:30.809914 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":135,"threads_started":1}
I20260810 18:56:30.812794 22144 tablet_service.cc:1467] Processing CreateTablet for tablet 5169aca46fa948e7a5bb8f85e173f6b5 (DEFAULT_TABLE table=test-table [id=d2219f2161a44a5383baf91604f549ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:30.812875 22589 tablet_service.cc:1467] Processing CreateTablet for tablet 5169aca46fa948e7a5bb8f85e173f6b5 (DEFAULT_TABLE table=test-table [id=d2219f2161a44a5383baf91604f549ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:30.812875 22361 tablet_service.cc:1467] Processing CreateTablet for tablet 5169aca46fa948e7a5bb8f85e173f6b5 (DEFAULT_TABLE table=test-table [id=d2219f2161a44a5383baf91604f549ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:30.813877 22144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5169aca46fa948e7a5bb8f85e173f6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:30.813907 22589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5169aca46fa948e7a5bb8f85e173f6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:30.813903 22361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5169aca46fa948e7a5bb8f85e173f6b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:30.821213 22715 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Bootstrap starting.
I20260810 18:56:30.822079 22717 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Bootstrap starting.
I20260810 18:56:30.822306 22716 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Bootstrap starting.
I20260810 18:56:30.823702 22715 tablet_bootstrap.cc:654] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:30.824450 22717 tablet_bootstrap.cc:654] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:30.824563 22715 log.cc:824] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:30.824877 22716 tablet_bootstrap.cc:654] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:30.825766 22716 log.cc:824] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:30.825766 22717 log.cc:824] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:30.827112 22715 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: No bootstrap required, opened a new log
I20260810 18:56:30.827332 22715 ts_tablet_manager.cc:1397] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:56:30.827937 22716 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: No bootstrap required, opened a new log
I20260810 18:56:30.828150 22716 ts_tablet_manager.cc:1397] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:56:30.828400 22717 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: No bootstrap required, opened a new log
I20260810 18:56:30.828610 22717 ts_tablet_manager.cc:1397] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 18:56:30.831547 22716 raft_consensus.cc:348] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:30.831810 22716 raft_consensus.cc:374] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:30.831887 22716 raft_consensus.cc:729] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521687247152450cb8ae19adb7434746, State: Initialized, Role: FOLLOWER
I20260810 18:56:30.832345 22716 consensus_queue.cc:260] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:30.832806 22715 raft_consensus.cc:348] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:30.833009 22469 heartbeater.cc:503] Master 127.30.153.190:37857 was elected leader, sending a full tablet report...
I20260810 18:56:30.833108 22715 raft_consensus.cc:374] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:30.833199 22715 raft_consensus.cc:729] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78028d924414f698e690c07228ebd77, State: Initialized, Role: FOLLOWER
I20260810 18:56:30.833372 22716 ts_tablet_manager.cc:1428] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 18:56:30.833693 22715 consensus_queue.cc:260] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:30.834100 22717 raft_consensus.cc:348] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:30.834405 22717 raft_consensus.cc:374] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:30.834512 22717 raft_consensus.cc:729] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef84dc10a6b414aba7292e25221221f, State: Initialized, Role: FOLLOWER
I20260810 18:56:30.834779 22249 heartbeater.cc:503] Master 127.30.153.190:37857 was elected leader, sending a full tablet report...
I20260810 18:56:30.834791 22715 ts_tablet_manager.cc:1428] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260810 18:56:30.834967 22717 consensus_queue.cc:260] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:30.835628 22690 heartbeater.cc:503] Master 127.30.153.190:37857 was elected leader, sending a full tablet report...
I20260810 18:56:30.836004 22717 ts_tablet_manager.cc:1428] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260810 18:56:31.003966 22691 tablet.cc:2369] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:56:31.017735 22473 tablet.cc:2369] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:56:31.031878 22250 tablet.cc:2369] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:56:31.092414 22722 raft_consensus.cc:482] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:31.092695 22722 raft_consensus.cc:504] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:31.094127 22722 leader_election.cc:288] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e78028d924414f698e690c07228ebd77 (127.30.153.129:40599), 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
I20260810 18:56:31.098819 22711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:31.094603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:36349 (local address 127.30.153.131:43551)
0810 18:56:31.094749 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:31.094755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:31.094924 (+ 169us) server_negotiation.cc:407] Connection header received
0810 18:56:31.095188 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:31.095193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:31.095271 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:31.095351 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:31.095666 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.096138 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.097803 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.098195 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.098317 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:31.098374 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:31.098599 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 18:56:31.098687 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:56:31.098836 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:31.094391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:43551 (local address 127.30.153.130:36349)
0810 18:56:31.094820 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:31.094838 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:31.095010 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:31.095393 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:31.095395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:31.095409 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:31.095537 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:31.095546 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.096268 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.096272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:31.097663 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:31.097678 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.098332 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.098335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:31.098459 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:31.098483 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 18:56:31.098554 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 18:56:31.098631 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":97,"thread_start_us":93,"threads_started":1}
I20260810 18:56:31.098865 22707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:31.094783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:60531 (local address 127.30.153.129:40599)
0810 18:56:31.094928 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:31.094932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:31.094948 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:56:31.095084 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:31.095089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:31.095145 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:31.095236 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:31.095582 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.096121 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.097801 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.098212 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.098351 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:31.098405 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:31.098670 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 18:56:31.098755 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 18:56:31.098851 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:31.094326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:40599 (local address 127.30.153.130:60531)
0810 18:56:31.094834 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:31.094852 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:31.094954 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:31.095272 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:31.095275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:31.095294 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:31.095454 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:31.095461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.096277 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.096281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:31.097643 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:31.097674 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.098367 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.098371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:31.098508 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:31.098555 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 18:56:31.098637 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 18:56:31.098702 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":31,"thread_start_us":114,"threads_started":1}
I20260810 18:56:31.099721 22614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5169aca46fa948e7a5bb8f85e173f6b5" candidate_uuid: "521687247152450cb8ae19adb7434746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef84dc10a6b414aba7292e25221221f" is_pre_election: true
I20260810 18:56:31.099951 22172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5169aca46fa948e7a5bb8f85e173f6b5" candidate_uuid: "521687247152450cb8ae19adb7434746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e78028d924414f698e690c07228ebd77" is_pre_election: true
I20260810 18:56:31.100124 22614 raft_consensus.cc:2454] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 521687247152450cb8ae19adb7434746 in term 0.
I20260810 18:56:31.100283 22172 raft_consensus.cc:2454] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 521687247152450cb8ae19adb7434746 in term 0.
I20260810 18:56:31.100715 22284 leader_election.cc:302] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f; no voters:
I20260810 18:56:31.101119 22722 raft_consensus.cc:2781] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:56:31.101250 22722 raft_consensus.cc:482] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:56:31.101322 22722 raft_consensus.cc:3037] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:31.102566 22722 raft_consensus.cc:504] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:31.102950 22722 leader_election.cc:288] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [CANDIDATE]: Term 1 election: Requested vote from peers e78028d924414f698e690c07228ebd77 (127.30.153.129:40599), 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
I20260810 18:56:31.103338 22614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5169aca46fa948e7a5bb8f85e173f6b5" candidate_uuid: "521687247152450cb8ae19adb7434746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ef84dc10a6b414aba7292e25221221f"
I20260810 18:56:31.103355 22172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5169aca46fa948e7a5bb8f85e173f6b5" candidate_uuid: "521687247152450cb8ae19adb7434746" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e78028d924414f698e690c07228ebd77"
I20260810 18:56:31.103489 22614 raft_consensus.cc:3037] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:31.103489 22172 raft_consensus.cc:3037] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:31.104918 22614 raft_consensus.cc:2454] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 521687247152450cb8ae19adb7434746 in term 1.
I20260810 18:56:31.104918 22172 raft_consensus.cc:2454] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 521687247152450cb8ae19adb7434746 in term 1.
I20260810 18:56:31.105296 22284 leader_election.cc:302] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f; no voters:
I20260810 18:56:31.105508 22722 raft_consensus.cc:2781] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:31.105742 22722 raft_consensus.cc:686] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 1 LEADER]: Becoming Leader. State: Replica: 521687247152450cb8ae19adb7434746, State: Running, Role: LEADER
I20260810 18:56:31.106309 22722 consensus_queue.cc:237] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:31.109416 21934 catalog_manager.cc:5495] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 reported cstate change: term changed from 0 to 1, leader changed from <none> to 521687247152450cb8ae19adb7434746 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "521687247152450cb8ae19adb7434746" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } health_report { overall_health: UNKNOWN } } }
I20260810 18:56:31.175181 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:31.171441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40992 (local address 127.30.153.130:34947)
0810 18:56:31.171630 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:31.171637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:31.171654 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:56:31.171755 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:31.171757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:31.171814 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:31.171936 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:56:31.172203 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.172718 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.173606 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:31.173832 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:31.173945 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:31.174315 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:56:31.174767 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:56:31.174904 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:31.174964 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 18:56:31.175021 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:56:31.181406 22614 raft_consensus.cc:1261] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 1 FOLLOWER]: Refusing update from remote peer 521687247152450cb8ae19adb7434746: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:31.181406 22172 raft_consensus.cc:1261] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Refusing update from remote peer 521687247152450cb8ae19adb7434746: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:31.182184 22729 consensus_queue.cc:1035] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:31.182354 22722 consensus_queue.cc:1035] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:31.190991 22734 mvcc.cc:205] Tried to move back new op lower bound from 7317046031071166464 to 7317046030774091776. Current Snapshot: MvccSnapshot[applied={T|T < 7317046031071166464}]
I20260810 18:56:31.191186 22737 mvcc.cc:205] Tried to move back new op lower bound from 7317046031071166464 to 7317046030774091776. Current Snapshot: MvccSnapshot[applied={T|T < 7317046031071166464}]
I20260810 18:56:31.198923 22735 mvcc.cc:205] Tried to move back new op lower bound from 7317046031071166464 to 7317046030774091776. Current Snapshot: MvccSnapshot[applied={T|T < 7317046031071166464 or (T in {7317046031071166464})}]
I20260810 18:56:31.199388 21932 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:31.199573 21932 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:31.202355 21932 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Sending DeleteTablet for 3 replicas of tablet 5169aca46fa948e7a5bb8f85e173f6b5
I20260810 18:56:31.203275 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 22033
I20260810 18:56:31.203208 22589 tablet_service.cc:1514] Processing DeleteTablet for tablet 5169aca46fa948e7a5bb8f85e173f6b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:31 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:49772
I20260810 18:56:31.203315 22144 tablet_service.cc:1514] Processing DeleteTablet for tablet 5169aca46fa948e7a5bb8f85e173f6b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:31 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:49764
I20260810 18:56:31.212792 22754 tablet_replica.cc:330] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: stopping tablet replica
I20260810 18:56:31.212750 22361 tablet_service.cc:1514] Processing DeleteTablet for tablet 5169aca46fa948e7a5bb8f85e173f6b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:31 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:40984
I20260810 18:56:31.213275 22759 tablet_replica.cc:330] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: stopping tablet replica
I20260810 18:56:31.213330 22754 raft_consensus.cc:2229] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:56:31.213634 22759 raft_consensus.cc:2229] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:56:31.213780 22754 raft_consensus.cc:2258] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:31.214149 22759 raft_consensus.cc:2258] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:31.215580 22754 ts_tablet_manager.cc:1905] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:31.218968 22759 ts_tablet_manager.cc:1905] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:31.219388 22754 ts_tablet_manager.cc:1918] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:31.219569 22754 log.cc:1196] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/5169aca46fa948e7a5bb8f85e173f6b5
I20260810 18:56:31.219996 22754 ts_tablet_manager.cc:1939] T 5169aca46fa948e7a5bb8f85e173f6b5 P 8ef84dc10a6b414aba7292e25221221f: Deleting consensus metadata
W20260810 18:56:31.220475 21914 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:40599 (error 108)
I20260810 18:56:31.221422 21908 catalog_manager.cc:4841] TS 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551): tablet 5169aca46fa948e7a5bb8f85e173f6b5 (table test-table [id=d2219f2161a44a5383baf91604f549ce]) successfully deleted
I20260810 18:56:31.221549 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 22255
I20260810 18:56:31.221940 22759 ts_tablet_manager.cc:1918] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:31.231215 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:31.230899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:40599 (local address 127.0.0.1:49766)
0810 18:56:31.231043 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:31.231133 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:40599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 18:56:31.231392 21914 catalog_manager.cc:4584] TS e78028d924414f698e690c07228ebd77 (127.30.153.129:40599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5169aca46fa948e7a5bb8f85e173f6b5: Network error: Client connection negotiation failed: client connection to 127.30.153.129:40599: connect: Connection refused (error 111)
I20260810 18:56:31.237453 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 22478
I20260810 18:56:31.239779 22709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:31.237662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:34947 (local address 127.0.0.1:40996)
0810 18:56:31.239578 (+ 1916us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:31.239640 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.130:34947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1789}
I20260810 18:56:31.249382 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 21877
I20260810 18:56:51.260635 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:37857
--webserver_interface=127.30.153.190
--webserver_port=43891
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:37857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:56:51.376995 22764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:51.377291 22764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:51.377393 22764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:51.382012 22764 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:56:51.382071 22764 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:56:51.382087 22764 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:56:51.382102 22764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:56:51.382124 22764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:51.382143 22764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:51.382166 22764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:56:51.382182 22764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:56:51.387481 22764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:37857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:37857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=43891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:51.388487 22764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:51.389437 22764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:56:51.403342 22764 server_base.cc:1034] running on GCE node
W20260810 18:56:51.403360 22771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:51.403311 22774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:51.403311 22779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:51.404294 22764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:51.405509 22764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:51.406675 22764 hybrid_clock.cc:648] HybridClock initialized: now 1786388211406662 us; error 31 us; skew 500 ppm
Aug 10 18:56:51 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388211, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:51.414901 22764 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:56:51.416014 22764 webserver.cc:460] Webserver started at http://127.30.153.190:43891/ using document root <none> and password file <none>
I20260810 18:56:51.416695 22764 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:51.416823 22764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:51.421286 22764 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:56:51.423983 22791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:51.425428 22764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:56:51.425594 22764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "607d12b05f9c4034b63f2dd8b072ec4a"
format_stamp: "Formatted at 2026-08-10 18:56:29 on dist-test-slave-68xk"
server_key: "a0d427b1605b507c88bfe0df527ed5f4"
server_key_iv: "6038b2a9ddc2ce6d7ba0426edc4b314d"
server_key_version: "encryptionkey@0"
I20260810 18:56:51.426239 22764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:51.437546 22764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:51.440760 22764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:51.440987 22764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:51.448745 22764 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:37857
I20260810 18:56:51.449230 22871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:37857 every 8 connection(s)
I20260810 18:56:51.450280 22764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:56:51.454828 22872 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:56:51.456449 22872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Bootstrap starting.
I20260810 18:56:51.459213 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 22764
I20260810 18:56:51.459687 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:40599
--local_ip_for_outbound_sockets=127.30.153.129
--tserver_master_addrs=127.30.153.190:37857
--webserver_port=43685
--webserver_interface=127.30.153.129
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:51.474766 22872 log.cc:824] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:51.483875 22872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:56:51.484292 22872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Bootstrap complete.
I20260810 18:56:51.488750 22872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:56:51.489409 22872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607d12b05f9c4034b63f2dd8b072ec4a, State: Initialized, Role: FOLLOWER
I20260810 18:56:51.489948 22872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:56:51.490092 22872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:56:51.490191 22872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:56:51.490298 22872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 1 FOLLOWER]: Advancing to term 2
I20260810 18:56:51.492256 22872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:56:51.492605 22872 leader_election.cc:302] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 607d12b05f9c4034b63f2dd8b072ec4a; no voters:
I20260810 18:56:51.492967 22872 leader_election.cc:288] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 18:56:51.493146 22878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 2 FOLLOWER]: Leader election won for term 2
I20260810 18:56:51.493424 22878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 2 LEADER]: Becoming Leader. State: Replica: 607d12b05f9c4034b63f2dd8b072ec4a, State: Running, Role: LEADER
I20260810 18:56:51.493752 22878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:56:51.494268 22872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:56:51.495275 22879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } } }
I20260810 18:56:51.495421 22879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:51.495790 22880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 607d12b05f9c4034b63f2dd8b072ec4a. Latest consensus state: current_term: 2 leader_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } } }
I20260810 18:56:51.495937 22880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: This master's current role is: LEADER
I20260810 18:56:51.496315 22888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:56:51.500384 22888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:56:51.500926 22888 catalog_manager.cc:1259] Loaded cluster ID: 0d144b340b934498be8ff28af848ba40
I20260810 18:56:51.501005 22888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:56:51.502852 22888 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:56:51.503460 22888 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Loaded TSK: 0
I20260810 18:56:51.513902 22888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Generated new TSK 1
I20260810 18:56:51.514195 22888 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:56:51.588032 22876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:51.588330 22876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:51.588405 22876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:51.592438 22876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:51.592525 22876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:51.592620 22876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:56:51.598142 22876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:40599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=43685
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:51.599416 22876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:51.600459 22876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:51.610389 22917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:51.610488 22916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:51.610389 22919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:51.611150 22876 server_base.cc:1034] running on GCE node
I20260810 18:56:51.611585 22876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:51.612313 22876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:51.613531 22876 hybrid_clock.cc:648] HybridClock initialized: now 1786388211613519 us; error 32 us; skew 500 ppm
Aug 10 18:56:51 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388211, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:51.616572 22876 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:56:51.617576 22876 webserver.cc:460] Webserver started at http://127.30.153.129:43685/ using document root <none> and password file <none>
I20260810 18:56:51.618168 22876 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:51.618247 22876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:51.622309 22876 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:56:51.624634 22933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:51.625957 22876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:56:51.626096 22876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "e78028d924414f698e690c07228ebd77"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "f874ed7e4518371edcdc19a5eeff1404"
server_key_iv: "520ad07a0cc240b0b9dad453573eabb0"
server_key_version: "encryptionkey@0"
I20260810 18:56:51.626703 22876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:51.659559 22876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:51.663046 22876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:51.663269 22876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:51.663938 22876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:51.665077 22953 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 18:56:51.669242 22876 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:56:51.669322 22876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260810 18:56:51.669404 22876 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:56:51.671089 22876 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:56:51.671164 22876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260810 18:56:51.671440 22953 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Bootstrap starting.
I20260810 18:56:51.682106 22876 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:40599
I20260810 18:56:51.682174 23123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:40599 every 8 connection(s)
I20260810 18:56:51.683172 22876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:56:51.688122 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 22876
I20260810 18:56:51.688694 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:34947
--local_ip_for_outbound_sockets=127.30.153.130
--tserver_master_addrs=127.30.153.190:37857
--webserver_port=39011
--webserver_interface=127.30.153.130
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:56:51.691802 22953 log.cc:824] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:56:51 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388211, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:51.702714 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:51.685488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:39249)
0810 18:56:51.686121 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:51.686165 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:51.687073 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:51.688457 (+ 1384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:51.688467 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:51.689068 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:51.689791 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:51.689813 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.691507 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.691510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:51.692649 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:51.692663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.693528 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.693532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:51.693741 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:51.694389 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:51.694406 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:51.696277 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:51.698987 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:51.698994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:51.699009 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:51.699345 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:51.699800 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:51.699804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:51.699806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:51.699973 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:51.700677 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:51.700684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:51.700955 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 18:56:51.701174 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 18:56:51.701564 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":162,"threads_started":1}
I20260810 18:56:51.702698 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:51.685503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:39249 (local address 127.30.153.190:37857)
0810 18:56:51.685929 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:51.685950 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:51.686299 (+ 349us) server_negotiation.cc:407] Connection header received
0810 18:56:51.687511 (+ 1212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:51.687521 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:51.687903 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:51.688352 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:51.690025 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.691317 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.692820 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.693349 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.693549 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:51.696514 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:51.696520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:51.696535 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:51.696566 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:51.698817 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:51.699510 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:51.699516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:51.699520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:51.699675 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:51.700092 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:51.700096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:51.700098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:51.700510 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:51.700704 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:51.701327 (+ 623us) server_negotiation.cc:299] Negotiation successful
0810 18:56:51.701647 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":154,"threads_started":1}
I20260810 18:56:51.703255 22953 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:56:51.703867 22953 tablet_bootstrap.cc:492] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Bootstrap complete.
I20260810 18:56:51.705680 22953 ts_tablet_manager.cc:1397] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Time spent bootstrapping tablet: real 0.034s user 0.024s sys 0.008s
I20260810 18:56:51.705864 23124 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37857
I20260810 18:56:51.706167 23124 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:51.706837 23124 heartbeater.cc:511] Master 127.30.153.190:37857 requested a full tablet report, sending...
I20260810 18:56:51.708855 22953 raft_consensus.cc:348] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:51.709400 22953 raft_consensus.cc:729] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78028d924414f698e690c07228ebd77, State: Initialized, Role: FOLLOWER
I20260810 18:56:51.709533 22816 ts_manager.cc:194] Registered new tserver with Master: e78028d924414f698e690c07228ebd77 (127.30.153.129:40599)
I20260810 18:56:51.709918 22953 consensus_queue.cc:260] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } } peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } }
I20260810 18:56:51.711057 22953 ts_tablet_manager.cc:1428] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 18:56:51.713625 22816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:39249
I20260810 18:56:51.716440 23124 heartbeater.cc:503] Master 127.30.153.190:37857 was elected leader, sending a full tablet report...
Aug 10 18:56:51 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388211, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.129@KRBTEST.COM
I20260810 18:56:51.726214 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:51.713013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:40599 (local address 127.0.0.1:36856)
0810 18:56:51.714202 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:51.714255 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:51.714441 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:51.715213 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:51.715221 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:51.715706 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:51.715899 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:51.715909 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.718164 (+ 2255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.718167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:51.719185 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:51.719198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.719793 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.719797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:51.719911 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:51.720570 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:51.720586 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:51.722179 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:51.724306 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:51.724318 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:51.724324 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:51.724574 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:51.724904 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:51.724907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:51.724909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:51.725002 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:51.725514 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:51.725520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:51.725654 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 18:56:51.725808 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 18:56:51.726022 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"thread_start_us":142,"threads_started":1}
I20260810 18:56:51.726227 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:51.713197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.30.153.129:40599)
0810 18:56:51.713902 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:51.713923 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:51.714384 (+ 461us) server_negotiation.cc:407] Connection header received
0810 18:56:51.714622 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:51.714628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:51.714870 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:51.715061 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:51.716062 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.717044 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.719314 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.719655 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.719754 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:51.722338 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:51.722356 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:51.722366 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:51.722390 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:51.724170 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:51.724712 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:51.724718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:51.724719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:51.724801 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:51.725111 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:51.725116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:51.725117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:51.725387 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:51.725538 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:51.725899 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 18:56:51.726071 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":146,"threads_started":1}
I20260810 18:56:51.727286 23021 tablet_service.cc:1514] Processing DeleteTablet for tablet 5169aca46fa948e7a5bb8f85e173f6b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:31 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:36856
I20260810 18:56:51.728183 23138 tablet_replica.cc:330] stopping tablet replica
I20260810 18:56:51.728504 23138 raft_consensus.cc:2229] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:56:51.728775 23138 raft_consensus.cc:2258] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:51.729926 23138 ts_tablet_manager.cc:1905] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:51.733785 23138 ts_tablet_manager.cc:1918] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:51.733878 23138 log.cc:1196] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal/wals/5169aca46fa948e7a5bb8f85e173f6b5
I20260810 18:56:51.734169 23138 ts_tablet_manager.cc:1939] T 5169aca46fa948e7a5bb8f85e173f6b5 P e78028d924414f698e690c07228ebd77: Deleting consensus metadata
I20260810 18:56:51.735888 22796 catalog_manager.cc:4841] TS e78028d924414f698e690c07228ebd77 (127.30.153.129:40599): tablet 5169aca46fa948e7a5bb8f85e173f6b5 (table test-table [id=d2219f2161a44a5383baf91604f549ce]) successfully deleted
W20260810 18:56:51.814379 23131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:51.814718 23131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:51.814819 23131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:51.818810 23131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:51.818889 23131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:51.818975 23131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:56:51.824205 23131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:34947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=39011
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:51.825300 23131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:51.826361 23131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:51.836439 23148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:51.836481 23151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:51.836491 23146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:51.836707 23131 server_base.cc:1034] running on GCE node
I20260810 18:56:51.837270 23131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:51.837924 23131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:51.839084 23131 hybrid_clock.cc:648] HybridClock initialized: now 1786388211839050 us; error 51 us; skew 500 ppm
Aug 10 18:56:51 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388211, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:51.841913 23131 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:56:51.842900 23131 webserver.cc:460] Webserver started at http://127.30.153.130:39011/ using document root <none> and password file <none>
I20260810 18:56:51.843482 23131 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:51.843597 23131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:51.847628 23131 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:56:51.849846 23168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:51.851184 23131 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:56:51.851351 23131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "521687247152450cb8ae19adb7434746"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "0229377f8d3e58d4985a9d546326c01f"
server_key_iv: "4aef83d67f4761fb83b0a44e2ecb30df"
server_key_version: "encryptionkey@0"
I20260810 18:56:51.851929 23131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:51.876266 23131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:51.879741 23131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:51.879966 23131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:51.880630 23131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:51.881844 23181 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260810 18:56:51.885819 23181 ts_tablet_manager.cc:1852] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260810 18:56:51.886225 23181 ts_tablet_manager.cc:1905] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:51.890228 23181 ts_tablet_manager.cc:1918] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:51.890328 23181 log.cc:1196] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/5169aca46fa948e7a5bb8f85e173f6b5
I20260810 18:56:51.890735 23181 ts_tablet_manager.cc:1939] T 5169aca46fa948e7a5bb8f85e173f6b5 P 521687247152450cb8ae19adb7434746: Deleting consensus metadata
I20260810 18:56:51.891021 23131 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260810 18:56:51.891103 23131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260810 18:56:51.891211 23131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:51.891261 23131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:51.900964 23131 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:34947
I20260810 18:56:51.901007 23377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:34947 every 8 connection(s)
I20260810 18:56:51.901850 23131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:56:51.908427 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 23131
I20260810 18:56:51.908910 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:43551
--local_ip_for_outbound_sockets=127.30.153.131
--tserver_master_addrs=127.30.153.190:37857
--webserver_port=45651
--webserver_interface=127.30.153.131
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 18:56:51 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388211, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:51.916975 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:51.903640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:46947 (local address 127.30.153.190:37857)
0810 18:56:51.903788 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:51.903792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:51.904652 (+ 860us) server_negotiation.cc:407] Connection header received
0810 18:56:51.905650 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:51.905653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:51.905704 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:51.905784 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:51.907022 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.907563 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.908741 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.908990 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.909118 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:51.912183 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:51.912187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:51.912192 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:51.912229 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:51.914156 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:51.914988 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:51.914994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:51.914999 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:51.915108 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:51.915559 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:51.915565 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:51.915569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:51.915864 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:51.915982 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:51.916541 (+ 559us) server_negotiation.cc:299] Negotiation successful
0810 18:56:51.916712 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:56:51.917608 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:51.903961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:46947)
0810 18:56:51.904492 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:51.904535 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:51.905425 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:51.905944 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:51.905951 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:51.906313 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:51.906853 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:51.906866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.907698 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.907700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:51.908585 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:51.908599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:51.909281 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:51.909288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:51.909525 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:51.910211 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:51.910236 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:51.911988 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:51.914327 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:51.914337 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:51.914358 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:51.914867 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:51.915262 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:51.915265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:51.915266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:51.915435 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:51.916003 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:51.916008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:51.916267 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 18:56:51.916426 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 18:56:51.916683 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":123,"threads_started":1}
I20260810 18:56:51.918864 23378 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37857
I20260810 18:56:51.919132 23378 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:51.919642 23378 heartbeater.cc:511] Master 127.30.153.190:37857 requested a full tablet report, sending...
I20260810 18:56:51.920782 22816 ts_manager.cc:194] Registered new tserver with Master: 521687247152450cb8ae19adb7434746 (127.30.153.130:34947)
I20260810 18:56:51.921382 22816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:46947
W20260810 18:56:52.026023 23383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:56:52.026317 23383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:56:52.026445 23383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:56:52.030627 23383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:56:52.030705 23383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:56:52.030792 23383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:56:52.036092 23383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:43551
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=45651
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:56:52.037127 23383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:56:52.038139 23383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:56:52.047863 23391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:52.047865 23392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:56:52.047837 23395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:56:52.048417 23383 server_base.cc:1034] running on GCE node
I20260810 18:56:52.048790 23383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:56:52.049465 23383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:56:52.050628 23383 hybrid_clock.cc:648] HybridClock initialized: now 1786388212050611 us; error 34 us; skew 500 ppm
Aug 10 18:56:52 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388212, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:56:52.053369 23383 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:56:52.054257 23383 webserver.cc:460] Webserver started at http://127.30.153.131:45651/ using document root <none> and password file <none>
I20260810 18:56:52.054960 23383 fs_manager.cc:359] Metadata directory not provided
I20260810 18:56:52.055042 23383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:56:52.058976 23383 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260810 18:56:52.061121 23407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:52.062306 23383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:56:52.062453 23383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "8ef84dc10a6b414aba7292e25221221f"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "1473bfb3ebd8984a55c5cc81e7885560"
server_key_iv: "9ec100c3cd49fbb61aa415d52ccce7a2"
server_key_version: "encryptionkey@0"
I20260810 18:56:52.062978 23383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:56:52.080093 23383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:56:52.083408 23383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:56:52.083630 23383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:56:52.084317 23383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:56:52.085202 23383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:56:52.085261 23383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:52.085304 23383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:56:52.085366 23383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:56:52.095710 23383 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:43551
I20260810 18:56:52.095739 23585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:43551 every 8 connection(s)
I20260810 18:56:52.096801 23383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:56:52.097632 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 23383
I20260810 18:56:52.107616 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.098389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.30.153.190:37857)
0810 18:56:52.098570 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.098575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.098878 (+ 303us) server_negotiation.cc:407] Connection header received
0810 18:56:52.099013 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.099016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.099072 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.099146 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:52.100143 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.101008 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.101888 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.102248 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.102361 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.103858 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:52.103865 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:52.103870 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:52.103903 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:52.105682 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:52.106228 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:52.106232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:52.106234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:52.106331 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:52.106677 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:52.106680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:52.106682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:52.106929 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:52.107119 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:52.107349 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 18:56:52.107450 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 10 18:56:52 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388212, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:56:52.111805 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.098832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:39183 (local address 127.30.153.190:37857)
0810 18:56:52.099320 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.099322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.099995 (+ 673us) server_negotiation.cc:407] Connection header received
0810 18:56:52.101607 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.101610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.101662 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.101748 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:52.103257 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.103843 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.104981 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.105214 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.105301 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.108299 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:52.108302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:52.108305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:52.108328 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:52.109936 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:52.110507 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:52.110510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:52.110511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:52.110589 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:52.110892 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:52.110894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:52.110895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:52.111048 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:52.111126 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:52.111590 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 18:56:52.111685 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":91,"threads_started":1}
I20260810 18:56:52.112694 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.099336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:39183)
0810 18:56:52.099858 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:52.099899 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:52.101410 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:52.101927 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:52.101936 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:52.102476 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:56:52.103111 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.103128 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.103970 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.103973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.104845 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.104854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.105378 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.105384 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.105620 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.106237 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:56:52.106256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:56:52.108159 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:56:52.110043 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:52.110048 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:52.110060 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:52.110348 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:52.110674 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:56:52.110676 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:56:52.110677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:56:52.110807 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:56:52.111163 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:56:52.111168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:56:52.111386 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 18:56:52.111542 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 18:56:52.111829 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":171,"threads_started":1}
I20260810 18:56:52.113801 23587 heartbeater.cc:347] Connected to a master server at 127.30.153.190:37857
I20260810 18:56:52.114048 23587 heartbeater.cc:464] Registering TS with master...
I20260810 18:56:52.114596 23587 heartbeater.cc:511] Master 127.30.153.190:37857 requested a full tablet report, sending...
I20260810 18:56:52.115651 22816 ts_manager.cc:194] Registered new tserver with Master: 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
I20260810 18:56:52.116250 22816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:39183
I20260810 18:56:52.121236 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:56:52.125151 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.121844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53292 (local address 127.30.153.190:37857)
0810 18:56:52.121969 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.121972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.122171 (+ 199us) server_negotiation.cc:407] Connection header received
0810 18:56:52.122284 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.122286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.122375 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.122511 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:56:52.122707 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.123180 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.123988 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.124207 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.124288 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.124637 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:56:52.124763 (+ 126us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 18:56:52.125004 (+ 241us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53292: token expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 18:56:52.125213 23590 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53292: token expired
W20260810 18:56:52.125536 22693 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.153.190:37857)
I20260810 18:56:52.133170 23590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.125911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53308 (local address 127.30.153.190:37857)
0810 18:56:52.126037 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.126041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.126054 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:56:52.126131 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.126133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.126169 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.126267 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:56:52.126943 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.127421 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.128221 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.128441 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.128547 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.129981 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:56:52.129984 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:56:52.129987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:56:52.130012 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:56:52.131496 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:52.131942 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:52.131945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:52.131947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:52.132029 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:56:52.132330 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:56:52.132333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:56:52.132334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:56:52.132503 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:56:52.132647 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:52.132889 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 18:56:52.133011 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:56:52.135881 22816 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:56:52.139196 22816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:56:52.148320 23021 tablet_service.cc:1467] Processing CreateTablet for tablet 24abec6815b048e59aa61d0bc98ffbd5 (DEFAULT_TABLE table=test-table [id=8b5a9619f62d4becbb4e4eb759e7f4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:52.149030 23021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24abec6815b048e59aa61d0bc98ffbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:52.151932 22953 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Bootstrap starting.
I20260810 18:56:52.151984 23136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.146267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:34947 (local address 127.0.0.1:59158)
0810 18:56:52.146469 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:52.146485 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:52.146570 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:52.147542 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:52.147546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:52.147566 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:52.147682 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.147687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.149015 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.149020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.150350 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.150361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.151500 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.151504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.151608 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.151662 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 18:56:52.151755 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:56:52.151844 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":55}
I20260810 18:56:52.152107 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.146647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.30.153.130:34947)
0810 18:56:52.147016 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.147025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.147047 (+ 22us) server_negotiation.cc:407] Connection header received
0810 18:56:52.147113 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.147117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.147371 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.147550 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:52.147796 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.148893 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.150513 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.151350 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.151485 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.151650 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:52.151809 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 18:56:52.151914 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":160,"threads_started":1}
I20260810 18:56:52.152833 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.147002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58360 (local address 127.30.153.131:43551)
0810 18:56:52.147344 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.147356 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.147377 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:56:52.147457 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.147463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.147655 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.147823 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:52.148248 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.149369 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.150838 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.151680 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.151805 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.151925 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:52.152588 (+ 663us) server_negotiation.cc:299] Negotiation successful
0810 18:56:52.152686 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":114,"threads_started":1}
I20260810 18:56:52.152863 22953 tablet_bootstrap.cc:654] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:52.153048 23603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.146434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:43551 (local address 127.0.0.1:58360)
0810 18:56:52.146753 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:52.146765 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:52.146829 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:52.147810 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:52.147813 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:52.147833 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:52.148079 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.148087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.149505 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.149508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.150685 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.150695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.152317 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.152326 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.152429 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.152449 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:56:52.152542 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:56:52.152617 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":4,"thread_start_us":106,"threads_started":1}
I20260810 18:56:52.153651 23254 tablet_service.cc:1467] Processing CreateTablet for tablet 24abec6815b048e59aa61d0bc98ffbd5 (DEFAULT_TABLE table=test-table [id=8b5a9619f62d4becbb4e4eb759e7f4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:52.154858 23254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24abec6815b048e59aa61d0bc98ffbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:52.155128 22953 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: No bootstrap required, opened a new log
I20260810 18:56:52.155241 22953 ts_tablet_manager.cc:1397] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260810 18:56:52.155694 22953 raft_consensus.cc:348] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.155865 22953 raft_consensus.cc:374] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:52.155923 22953 raft_consensus.cc:729] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78028d924414f698e690c07228ebd77, State: Initialized, Role: FOLLOWER
I20260810 18:56:52.156061 22953 consensus_queue.cc:260] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.156351 22953 ts_tablet_manager.cc:1428] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260810 18:56:52.158880 23487 tablet_service.cc:1467] Processing CreateTablet for tablet 24abec6815b048e59aa61d0bc98ffbd5 (DEFAULT_TABLE table=test-table [id=8b5a9619f62d4becbb4e4eb759e7f4bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:56:52.159991 23487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24abec6815b048e59aa61d0bc98ffbd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:56:52.160112 23181 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: Bootstrap starting.
I20260810 18:56:52.162048 23181 tablet_bootstrap.cc:654] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:52.163066 23181 log.cc:824] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:52.164883 23181 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: No bootstrap required, opened a new log
I20260810 18:56:52.165096 23181 ts_tablet_manager.cc:1397] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 18:56:52.167053 23616 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: Bootstrap starting.
I20260810 18:56:52.168525 23181 raft_consensus.cc:348] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.168809 23181 raft_consensus.cc:374] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:52.168869 23181 raft_consensus.cc:729] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521687247152450cb8ae19adb7434746, State: Initialized, Role: FOLLOWER
I20260810 18:56:52.169333 23181 consensus_queue.cc:260] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.169507 23616 tablet_bootstrap.cc:654] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: Neither blocks nor log segments found. Creating new log.
I20260810 18:56:52.170053 23378 heartbeater.cc:503] Master 127.30.153.190:37857 was elected leader, sending a full tablet report...
I20260810 18:56:52.170295 23616 log.cc:824] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:56:52.170365 23181 ts_tablet_manager.cc:1428] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260810 18:56:52.172438 23616 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: No bootstrap required, opened a new log
I20260810 18:56:52.172662 23616 ts_tablet_manager.cc:1397] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 18:56:52.175760 23616 raft_consensus.cc:348] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.176043 23616 raft_consensus.cc:374] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:56:52.176101 23616 raft_consensus.cc:729] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ef84dc10a6b414aba7292e25221221f, State: Initialized, Role: FOLLOWER
I20260810 18:56:52.176574 23616 consensus_queue.cc:260] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.177320 23587 heartbeater.cc:503] Master 127.30.153.190:37857 was elected leader, sending a full tablet report...
I20260810 18:56:52.177592 23616 ts_tablet_manager.cc:1428] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260810 18:56:52.184520 23125 tablet.cc:2369] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:56:52.277101 23625 raft_consensus.cc:482] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:56:52.277339 23625 raft_consensus.cc:504] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.278730 23625 leader_election.cc:288] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 521687247152450cb8ae19adb7434746 (127.30.153.130:34947), e78028d924414f698e690c07228ebd77 (127.30.153.129:40599)
I20260810 18:56:52.283032 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.279073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:34947 (local address 127.30.153.131:40527)
0810 18:56:52.279205 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:52.279222 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:52.279327 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:52.279641 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:52.279644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:52.279666 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:52.279803 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.279809 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.280817 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.280819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.281866 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.281876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.282653 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.282655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.282735 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.282766 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:56:52.282828 (+ 62us) client_negotiation.cc:240] Negotiation successful
0810 18:56:52.282882 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 18:56:52.283056 23610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.279231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:40527 (local address 127.30.153.130:34947)
0810 18:56:52.279379 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.279384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.279402 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:56:52.279463 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.279466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.279522 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.279625 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:52.279937 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.280697 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.282003 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.282546 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.282668 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.282721 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:52.282865 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 18:56:52.282930 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:56:52.283696 23291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24abec6815b048e59aa61d0bc98ffbd5" candidate_uuid: "8ef84dc10a6b414aba7292e25221221f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521687247152450cb8ae19adb7434746" is_pre_election: true
I20260810 18:56:52.283840 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.279362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:40599 (local address 127.30.153.131:54883)
0810 18:56:52.279834 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:52.279846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:56:52.279931 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:56:52.280339 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:56:52.280341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:56:52.280354 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:56:52.280449 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.280453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.281296 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.281300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.282287 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:56:52.282295 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.283388 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.283392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:56:52.283505 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.283533 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:56:52.283634 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:56:52.283700 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":171,"threads_started":1}
I20260810 18:56:52.283870 23629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.279527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:54883 (local address 127.30.153.129:40599)
0810 18:56:52.280064 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.280069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.280083 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:56:52.280135 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.280138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.280201 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.280320 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:56:52.280571 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.281172 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.282408 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.283102 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.283227 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.283391 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:52.283658 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 18:56:52.283741 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":91,"threads_started":1}
I20260810 18:56:52.284066 23291 raft_consensus.cc:2454] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef84dc10a6b414aba7292e25221221f in term 0.
I20260810 18:56:52.284340 23053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24abec6815b048e59aa61d0bc98ffbd5" candidate_uuid: "8ef84dc10a6b414aba7292e25221221f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e78028d924414f698e690c07228ebd77" is_pre_election: true
I20260810 18:56:52.284571 23053 raft_consensus.cc:2454] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ef84dc10a6b414aba7292e25221221f in term 0.
I20260810 18:56:52.284570 23415 leader_election.cc:302] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f; no voters:
I20260810 18:56:52.284921 23625 raft_consensus.cc:2781] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:56:52.285017 23625 raft_consensus.cc:482] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:56:52.285079 23625 raft_consensus.cc:3037] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:52.286149 23625 raft_consensus.cc:504] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.286531 23625 leader_election.cc:288] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [CANDIDATE]: Term 1 election: Requested vote from peers 521687247152450cb8ae19adb7434746 (127.30.153.130:34947), e78028d924414f698e690c07228ebd77 (127.30.153.129:40599)
I20260810 18:56:52.286895 23291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24abec6815b048e59aa61d0bc98ffbd5" candidate_uuid: "8ef84dc10a6b414aba7292e25221221f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "521687247152450cb8ae19adb7434746"
I20260810 18:56:52.286965 23053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24abec6815b048e59aa61d0bc98ffbd5" candidate_uuid: "8ef84dc10a6b414aba7292e25221221f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e78028d924414f698e690c07228ebd77"
I20260810 18:56:52.287019 23291 raft_consensus.cc:3037] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:52.287147 23053 raft_consensus.cc:3037] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:56:52.288108 23291 raft_consensus.cc:2454] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef84dc10a6b414aba7292e25221221f in term 1.
I20260810 18:56:52.288182 23053 raft_consensus.cc:2454] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ef84dc10a6b414aba7292e25221221f in term 1.
I20260810 18:56:52.288414 23415 leader_election.cc:302] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f; no voters:
I20260810 18:56:52.288587 23625 raft_consensus.cc:2781] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:56:52.288841 23625 raft_consensus.cc:686] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 1 LEADER]: Becoming Leader. State: Replica: 8ef84dc10a6b414aba7292e25221221f, State: Running, Role: LEADER
I20260810 18:56:52.289166 23625 consensus_queue.cc:237] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:56:52.292348 22816 catalog_manager.cc:5495] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ef84dc10a6b414aba7292e25221221f (127.30.153.131). New cstate: current_term: 1 leader_uuid: "8ef84dc10a6b414aba7292e25221221f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } health_report { overall_health: UNKNOWN } } }
I20260810 18:56:52.305471 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.301821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58366 (local address 127.30.153.131:43551)
0810 18:56:52.301973 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 18:56:52.301978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:56:52.301995 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:56:52.302120 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:56:52.302123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:56:52.302197 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:56:52.302336 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:56:52.302615 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.303111 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.304003 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:56:52.304227 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:56:52.304337 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:56:52.304666 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:56:52.305088 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:56:52.305235 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:56:52.305272 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 18:56:52.305328 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 18:56:52.311558 23053 raft_consensus.cc:1261] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Refusing update from remote peer 8ef84dc10a6b414aba7292e25221221f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:52.311558 23291 raft_consensus.cc:1261] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 1 FOLLOWER]: Refusing update from remote peer 8ef84dc10a6b414aba7292e25221221f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:56:52.312314 23625 consensus_queue.cc:1035] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:52.312477 23630 consensus_queue.cc:1035] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [LEADER]: Connected to new peer: Peer: permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:56:52.320809 23637 mvcc.cc:205] Tried to move back new op lower bound from 7317046117620768768 to 7317046117539225600. Current Snapshot: MvccSnapshot[applied={T|T < 7317046117620768768}]
I20260810 18:56:52.328542 22816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:52.328750 22816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:56:52.331350 22816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Sending DeleteTablet for 3 replicas of tablet 24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:56:52.332060 23021 tablet_service.cc:1514] Processing DeleteTablet for tablet 24abec6815b048e59aa61d0bc98ffbd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:52 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:36856
I20260810 18:56:52.332078 23254 tablet_service.cc:1514] Processing DeleteTablet for tablet 24abec6815b048e59aa61d0bc98ffbd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:52 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:59158
I20260810 18:56:52.332181 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 22876
I20260810 18:56:52.340461 23487 tablet_service.cc:1514] Processing DeleteTablet for tablet 24abec6815b048e59aa61d0bc98ffbd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:56:52 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:58360
I20260810 18:56:52.342887 23649 tablet_replica.cc:330] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: stopping tablet replica
I20260810 18:56:52.343360 23649 raft_consensus.cc:2229] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:56:52.343398 23648 tablet_replica.cc:330] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: stopping tablet replica
I20260810 18:56:52.343886 23649 raft_consensus.cc:2258] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:52.343895 23648 raft_consensus.cc:2229] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:56:52.344360 23648 raft_consensus.cc:2258] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:56:52.345746 23649 ts_tablet_manager.cc:1905] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:52.346310 23648 ts_tablet_manager.cc:1905] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:56:52.349277 23648 ts_tablet_manager.cc:1918] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:52.349465 23648 log.cc:1196] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:56:52.349522 23649 ts_tablet_manager.cc:1918] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:56:52.349637 23649 log.cc:1196] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:56:52.349992 23648 ts_tablet_manager.cc:1939] T 24abec6815b048e59aa61d0bc98ffbd5 P 521687247152450cb8ae19adb7434746: Deleting consensus metadata
I20260810 18:56:52.350039 23649 ts_tablet_manager.cc:1939] T 24abec6815b048e59aa61d0bc98ffbd5 P 8ef84dc10a6b414aba7292e25221221f: Deleting consensus metadata
W20260810 18:56:52.350970 22796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:40599 (error 108)
I20260810 18:56:52.351405 22794 catalog_manager.cc:4841] TS 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551): tablet 24abec6815b048e59aa61d0bc98ffbd5 (table test-table [id=8b5a9619f62d4becbb4e4eb759e7f4bf]) successfully deleted
I20260810 18:56:52.351516 22796 catalog_manager.cc:4841] TS 521687247152450cb8ae19adb7434746 (127.30.153.130:34947): tablet 24abec6815b048e59aa61d0bc98ffbd5 (table test-table [id=8b5a9619f62d4becbb4e4eb759e7f4bf]) successfully deleted
I20260810 18:56:52.352452 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 23131
I20260810 18:56:52.352553 23603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:56:52.352210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:40599 (local address 127.0.0.1:36870)
0810 18:56:52.352376 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 18:56:52.352458 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:40599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 18:56:52.352741 22796 catalog_manager.cc:4584] TS e78028d924414f698e690c07228ebd77 (127.30.153.129:40599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24abec6815b048e59aa61d0bc98ffbd5: Network error: Client connection negotiation failed: client connection to 127.30.153.129:40599: connect: Connection refused (error 111)
I20260810 18:56:52.369302 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 23383
I20260810 18:56:52.385208 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 22764
I20260810 18:57:12.398103 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:37857
--webserver_interface=127.30.153.190
--webserver_port=43891
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:37857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:12.512651 23651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:12.512902 23651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:12.512941 23651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:12.516892 23651 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 18:57:12.516939 23651 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 18:57:12.516954 23651 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 18:57:12.516968 23651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:12.517014 23651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:12.517058 23651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:12.517087 23651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:12.517122 23651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:12.522214 23651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:37857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:37857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=43891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:12.523305 23651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:12.524308 23651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:12.533102 23658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:12.533175 23651 server_base.cc:1034] running on GCE node
W20260810 18:57:12.533134 23661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:12.533078 23659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:12.533877 23651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:12.535015 23651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:12.536221 23651 hybrid_clock.cc:648] HybridClock initialized: now 1786388232536206 us; error 29 us; skew 500 ppm
Aug 10 18:57:12 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388232, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:12.544572 23651 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:57:12.545500 23651 webserver.cc:460] Webserver started at http://127.30.153.190:43891/ using document root <none> and password file <none>
I20260810 18:57:12.546048 23651 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:12.546118 23651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:12.549975 23651 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:12.552640 23671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:12.554152 23651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 18:57:12.554316 23651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "607d12b05f9c4034b63f2dd8b072ec4a"
format_stamp: "Formatted at 2026-08-10 18:56:29 on dist-test-slave-68xk"
server_key: "a0d427b1605b507c88bfe0df527ed5f4"
server_key_iv: "6038b2a9ddc2ce6d7ba0426edc4b314d"
server_key_version: "encryptionkey@0"
I20260810 18:57:12.554987 23651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:12.574051 23651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:12.577239 23651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:12.577457 23651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:12.586393 23651 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:37857
I20260810 18:57:12.586728 23759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:37857 every 8 connection(s)
I20260810 18:57:12.587680 23651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:12.592245 23761 sys_catalog.cc:263] Verifying existing consensus state
I20260810 18:57:12.593986 23761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Bootstrap starting.
I20260810 18:57:12.596158 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 23651
I20260810 18:57:12.596701 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:40599
--local_ip_for_outbound_sockets=127.30.153.129
--tserver_master_addrs=127.30.153.190:37857
--webserver_port=43685
--webserver_interface=127.30.153.129
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:57:12.613770 23761 log.cc:824] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:12.625389 23761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:57:12.625980 23761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Bootstrap complete.
I20260810 18:57:12.630348 23761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:57:12.630932 23761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607d12b05f9c4034b63f2dd8b072ec4a, State: Initialized, Role: FOLLOWER
I20260810 18:57:12.631565 23761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:57:12.631705 23761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:12.631799 23761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:12.631925 23761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 2 FOLLOWER]: Advancing to term 3
I20260810 18:57:12.634074 23761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:57:12.634490 23761 leader_election.cc:302] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 607d12b05f9c4034b63f2dd8b072ec4a; no voters:
I20260810 18:57:12.634909 23761 leader_election.cc:288] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 18:57:12.634968 23769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 3 FOLLOWER]: Leader election won for term 3
I20260810 18:57:12.635226 23769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [term 3 LEADER]: Becoming Leader. State: Replica: 607d12b05f9c4034b63f2dd8b072ec4a, State: Running, Role: LEADER
I20260810 18:57:12.635543 23769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } }
I20260810 18:57:12.636291 23761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:12.636591 23771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } } }
I20260810 18:57:12.636739 23771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:12.636977 23770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 607d12b05f9c4034b63f2dd8b072ec4a. Latest consensus state: current_term: 3 leader_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607d12b05f9c4034b63f2dd8b072ec4a" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 37857 } } }
I20260810 18:57:12.637077 23770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:12.637156 23774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:12.642720 23774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:12.643527 23774 catalog_manager.cc:1259] Loaded cluster ID: 0d144b340b934498be8ff28af848ba40
I20260810 18:57:12.643608 23774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:12.645452 23774 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:12.646113 23774 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Loaded TSK: 0
I20260810 18:57:12.646209 23774 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 607d12b05f9c4034b63f2dd8b072ec4a: Loaded TSK: 1
F20260810 18:57:12.646888 23774 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f88ba0719ad google::LogMessage::Fail() at ??:0
@ 0x7f88ba07488a google::LogMessage::SendToLog() at ??:0
@ 0x7f88ba0714de google::LogMessage::Flush() at ??:0
@ 0x7f88ba072a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f88bcd3518d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f88bcd2df0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f88c49d67bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f88c49b099c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f88c49b1178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f88c49f2df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f88c4a1182b std::function<>::operator()() at ??:0
@ 0x7f88c49b0b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f88c49b1f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f88c49ae579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f88c49f2640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f88bc2c33d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f88bc2c3c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f88bc2c53f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f88bc2b4512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f88bd8f86db start_thread at ??:0
@ 0x7f88b8b6f61f clone at ??:0
W20260810 18:57:12.752800 23763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:12.753098 23763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:12.753201 23763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:12.757493 23763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:12.757586 23763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:12.757690 23763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:57:12.763278 23763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:40599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=43685
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:12.764392 23763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:12.765424 23763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:12.775102 23842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:12.775094 23840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:12.775095 23845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:12.775864 23763 server_base.cc:1034] running on GCE node
I20260810 18:57:12.776331 23763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:12.777038 23763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:12.778256 23763 hybrid_clock.cc:648] HybridClock initialized: now 1786388232778202 us; error 39 us; skew 500 ppm
Aug 10 18:57:12 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388232, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:12.781406 23763 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:57:12.782387 23763 webserver.cc:460] Webserver started at http://127.30.153.129:43685/ using document root <none> and password file <none>
I20260810 18:57:12.783054 23763 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:12.783124 23763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:12.787571 23763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:57:12.790014 23853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:12.791463 23763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:12.791610 23763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "e78028d924414f698e690c07228ebd77"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "f874ed7e4518371edcdc19a5eeff1404"
server_key_iv: "520ad07a0cc240b0b9dad453573eabb0"
server_key_version: "encryptionkey@0"
I20260810 18:57:12.792244 23763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:12.817495 23763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:12.820849 23763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:12.821069 23763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:12.821730 23763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:12.822929 23869 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260810 18:57:12.827319 23763 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 18:57:12.827390 23763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260810 18:57:12.827440 23763 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 18:57:12.829223 23763 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 18:57:12.829296 23763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260810 18:57:12.829555 23869 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Bootstrap starting.
I20260810 18:57:12.840969 23763 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:40599
I20260810 18:57:12.841187 24045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:40599 every 8 connection(s)
I20260810 18:57:12.842283 23763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:57:12.846038 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 23763
I20260810 18:57:12.846698 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:34947
--local_ip_for_outbound_sockets=127.30.153.130
--tserver_master_addrs=127.30.153.190:37857
--webserver_port=39011
--webserver_interface=127.30.153.130
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:57:12.851795 23869 log.cc:824] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:12.858896 23869 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 18:57:12.859331 23869 tablet_bootstrap.cc:492] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Bootstrap complete.
I20260810 18:57:12.860528 23869 ts_tablet_manager.cc:1397] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Time spent bootstrapping tablet: real 0.031s user 0.025s sys 0.004s
I20260810 18:57:12.863337 23869 raft_consensus.cc:348] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:57:12.863768 23869 raft_consensus.cc:729] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e78028d924414f698e690c07228ebd77, State: Initialized, Role: FOLLOWER
I20260810 18:57:12.864300 23869 consensus_queue.cc:260] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:57:12.865279 23869 ts_tablet_manager.cc:1428] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260810 18:57:12.906150 24050 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:57:12.844647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:50689)
0810 18:57:12.845291 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:12.845352 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:12.846529 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:12.905195 (+ 58666us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":203,"threads_started":1}
W20260810 18:57:12.906840 24046 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:37857 (0 consecutive failures): Network error: Failed to ping master at 127.30.153.190:37857: Client connection negotiation failed: client connection to 127.30.153.190:37857: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 18:57:12.907655 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:12.907360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:37657)
0810 18:57:12.907508 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:12.907586 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 18:57:12.908408 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:12.908130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:45515)
0810 18:57:12.908266 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:12.908330 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 18:57:12.908629 24046 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 18:57:12.980154 24051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:12.980393 24051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:12.980468 24051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:12.984516 24051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:12.984580 24051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:12.984671 24051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:57:12.989852 24051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:34947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=39011
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:12.990974 24051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:12.992005 24051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:13.002058 24065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:13.002139 24069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:13.002105 24067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:13.002673 24051 server_base.cc:1034] running on GCE node
I20260810 18:57:13.003120 24051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:13.003821 24051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:13.005000 24051 hybrid_clock.cc:648] HybridClock initialized: now 1786388233004980 us; error 28 us; skew 500 ppm
Aug 10 18:57:13 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388233, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:13.009091 24051 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:57:13.010262 24051 webserver.cc:460] Webserver started at http://127.30.153.130:39011/ using document root <none> and password file <none>
I20260810 18:57:13.011008 24051 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:13.011077 24051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:13.015731 24051 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:57:13.017863 24078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:13.019189 24051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:13.019332 24051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "521687247152450cb8ae19adb7434746"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "0229377f8d3e58d4985a9d546326c01f"
server_key_iv: "4aef83d67f4761fb83b0a44e2ecb30df"
server_key_version: "encryptionkey@0"
I20260810 18:57:13.019878 24051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:13.037735 24051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:13.040766 24051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:13.040972 24051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:13.041568 24051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:13.042402 24051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:13.042491 24051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:13.042560 24051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:13.042600 24051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:13.052055 24051 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:34947
I20260810 18:57:13.052088 24257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:34947 every 8 connection(s)
I20260810 18:57:13.053040 24051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:57:13.057188 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:13.055307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:55987)
0810 18:57:13.055795 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:13.055903 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":145,"threads_started":1}
W20260810 18:57:13.057686 24258 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:37857 (0 consecutive failures): Network error: Failed to ping master at 127.30.153.190:37857: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
I20260810 18:57:13.058363 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:13.058123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:37697)
0810 18:57:13.058262 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:13.058309 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 18:57:13.059001 24266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:13.058826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:41703)
0810 18:57:13.058915 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:13.058956 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 18:57:13.059216 24258 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:57:13.061328 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 24051
I20260810 18:57:13.061861 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:43551
--local_ip_for_outbound_sockets=127.30.153.131
--tserver_master_addrs=127.30.153.190:37857
--webserver_port=45651
--webserver_interface=127.30.153.131
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:13.177166 24267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:13.177407 24267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:13.177474 24267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:13.181556 24267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:13.181614 24267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:13.181670 24267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:57:13.186921 24267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:43551
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=45651
--tserver_master_addrs=127.30.153.190:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:13.187937 24267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:13.188947 24267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:13.198617 24277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:13.198611 24280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:13.198611 24276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:13.198849 24267 server_base.cc:1034] running on GCE node
I20260810 18:57:13.199326 24267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:13.199928 24267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:13.201095 24267 hybrid_clock.cc:648] HybridClock initialized: now 1786388233201082 us; error 27 us; skew 500 ppm
Aug 10 18:57:13 dist-test-slave-68xk krb5kdc[21859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388233, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:13.203804 24267 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:57:13.204644 24267 webserver.cc:460] Webserver started at http://127.30.153.131:45651/ using document root <none> and password file <none>
I20260810 18:57:13.205228 24267 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:13.205322 24267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:13.209102 24267 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:13.211349 24289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:13.212590 24267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:13.212739 24267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "8ef84dc10a6b414aba7292e25221221f"
format_stamp: "Formatted at 2026-08-10 18:56:30 on dist-test-slave-68xk"
server_key: "1473bfb3ebd8984a55c5cc81e7885560"
server_key_iv: "9ec100c3cd49fbb61aa415d52ccce7a2"
server_key_version: "encryptionkey@0"
I20260810 18:57:13.213320 24267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:13.243239 24267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:13.246312 24267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:13.246544 24267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:13.247152 24267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:13.248009 24267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:13.248080 24267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:13.248148 24267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:13.248195 24267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:13.257543 24267 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:43551
I20260810 18:57:13.257575 24463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:43551 every 8 connection(s)
I20260810 18:57:13.258601 24267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:57:13.259935 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 24267
I20260810 18:57:13.262744 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:13.260755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:34341)
0810 18:57:13.261243 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:13.261381 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":150,"threads_started":1}
W20260810 18:57:13.263223 24464 heartbeater.cc:644] Failed to heartbeat to 127.30.153.190:37857 (0 consecutive failures): Network error: Failed to ping master at 127.30.153.190:37857: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
I20260810 18:57:13.263831 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:13.263606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:42209)
0810 18:57:13.263717 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:13.263771 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 18:57:13.264428 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:13.264215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:38537)
0810 18:57:13.264317 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:13.264367 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 18:57:13.264603 24464 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 18:57:13.909767 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:13.909269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:56357)
0810 18:57:13.909567 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:13.909631 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":86,"threads_started":1}
I20260810 18:57:14.060161 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:14.059768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:46091)
0810 18:57:14.060003 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:14.060058 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":73,"threads_started":1}
I20260810 18:57:14.265578 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:14.265154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:40415)
0810 18:57:14.265421 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:14.265476 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":106,"threads_started":1}
I20260810 18:57:14.592173 24474 raft_consensus.cc:482] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:14.592427 24474 raft_consensus.cc:504] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:57:14.593674 24474 leader_election.cc:288] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 521687247152450cb8ae19adb7434746 (127.30.153.130:34947), 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
Aug 10 18:57:14 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388232, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.130@KRBTEST.COM
Aug 10 18:57:14 dist-test-slave-68xk krb5kdc[21859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388232, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.131@KRBTEST.COM
I20260810 18:57:14.610112 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:14.593946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:43551 (local address 127.30.153.129:56805)
0810 18:57:14.594548 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:14.594563 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:14.594668 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:14.596792 (+ 2124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:14.596798 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:14.597450 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:14.598281 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:14.598304 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:14.601237 (+ 2933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:14.601239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:14.601741 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:14.601747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:14.602544 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:14.602547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:14.602633 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:14.603113 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:14.603125 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:14.604706 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:14.607152 (+ 2446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:14.607157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:14.607162 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:14.607525 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:14.607931 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:14.607934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:14.607935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:14.608067 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:14.608925 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:14.608932 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:14.609351 (+ 419us) client_negotiation.cc:769] Sending connection context
0810 18:57:14.609594 (+ 243us) client_negotiation.cc:240] Negotiation successful
0810 18:57:14.609921 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":139,"spinlock_wait_cycles":51328,"thread_start_us":93,"threads_started":1}
I20260810 18:57:14.610126 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:14.594452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:56805 (local address 127.30.153.131:43551)
0810 18:57:14.594768 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:14.594798 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:14.594843 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:57:14.595860 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:14.595871 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:14.596326 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:14.596659 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:14.598502 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:14.600083 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:14.601859 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:14.602350 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:14.602593 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:14.604913 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:14.604918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:14.604929 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:14.604961 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:14.607053 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:14.607668 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:14.607675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:14.607681 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:14.607822 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:14.608198 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:14.608201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:14.608202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:14.608740 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:14.608886 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:14.609753 (+ 867us) server_negotiation.cc:299] Negotiation successful
0810 18:57:14.609986 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":121,"threads_started":1}
I20260810 18:57:14.610374 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:14.594323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:51277 (local address 127.30.153.130:34947)
0810 18:57:14.595124 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:14.595138 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:14.595179 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:57:14.595942 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:14.595949 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:14.596326 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:14.596658 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:14.598502 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:14.600064 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:14.601258 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:14.601731 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:14.601908 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:14.604689 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:14.604713 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:14.604725 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:14.604753 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:14.606914 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:14.607941 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:14.607946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:14.607949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:14.608062 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:14.608380 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:14.608384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:14.608385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:14.608813 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:14.608920 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:14.609866 (+ 946us) server_negotiation.cc:299] Negotiation successful
0810 18:57:14.610205 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":129,"threads_started":1}
I20260810 18:57:14.611536 24475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:14.593853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:34947 (local address 127.30.153.129:51277)
0810 18:57:14.594085 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:14.594117 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:14.594232 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:14.596792 (+ 2560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:14.596798 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:14.597450 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:14.598281 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:14.598304 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:14.600243 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:14.600246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:14.601114 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:14.601124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:14.601869 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:14.601873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:14.602050 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:14.602762 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:14.602779 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:14.604455 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:14.607036 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:14.607042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:14.607055 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:14.607520 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:14.608168 (+ 648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:14.608173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:14.608175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:14.608287 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:14.608932 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:14.608935 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:14.609350 (+ 415us) client_negotiation.cc:769] Sending connection context
0810 18:57:14.609592 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 18:57:14.611368 (+ 1776us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":1733,"spinlock_wait_cycles":4224,"thread_start_us":69,"threads_started":1}
W20260810 18:57:14.612756 23856 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
W20260810 18:57:14.613454 23861 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 521687247152450cb8ae19adb7434746 (127.30.153.130:34947): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:57:14.613569 23861 leader_election.cc:302] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e78028d924414f698e690c07228ebd77; no voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f
I20260810 18:57:14.613852 24474 raft_consensus.cc:2726] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:57:14.911037 24475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:14.910733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:37009)
0810 18:57:14.910883 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:14.910949 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 18:57:15.061415 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:15.060963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:44363)
0810 18:57:15.061245 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:15.061304 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
I20260810 18:57:15.266831 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:15.266301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:39819)
0810 18:57:15.266656 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:15.266733 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":75,"threads_started":1}
I20260810 18:57:15.912362 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:15.911869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:51991)
0810 18:57:15.912199 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:15.912261 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":134,"threads_started":1}
I20260810 18:57:16.062819 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:16.062297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:54593)
0810 18:57:16.062638 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:16.062722 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":154,"threads_started":1}
I20260810 18:57:16.268041 24491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:16.267602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:47893)
0810 18:57:16.267870 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:16.267930 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
I20260810 18:57:16.490367 24492 raft_consensus.cc:482] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:16.490641 24492 raft_consensus.cc:504] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:57:16.491220 24492 leader_election.cc:288] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 521687247152450cb8ae19adb7434746 (127.30.153.130:34947), 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
W20260810 18:57:16.492054 23861 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 521687247152450cb8ae19adb7434746 (127.30.153.130:34947): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
W20260810 18:57:16.492197 23856 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:57:16.492301 23856 leader_election.cc:302] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e78028d924414f698e690c07228ebd77; no voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f
I20260810 18:57:16.492501 24492 raft_consensus.cc:2726] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:57:16.913601 24493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:16.913150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:46659)
0810 18:57:16.913443 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:16.913502 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":84,"threads_started":1}
I20260810 18:57:17.064038 24494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:17.063587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:36893)
0810 18:57:17.063866 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:17.063924 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":78,"threads_started":1}
I20260810 18:57:17.269266 24495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:17.268844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:45111)
0810 18:57:17.269109 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:17.269163 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
I20260810 18:57:17.915153 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:17.914661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:56641)
0810 18:57:17.914979 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:17.915040 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":83,"threads_started":1}
I20260810 18:57:18.029747 24497 raft_consensus.cc:482] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:18.029945 24497 raft_consensus.cc:504] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:57:18.030488 24497 leader_election.cc:288] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 521687247152450cb8ae19adb7434746 (127.30.153.130:34947), 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
W20260810 18:57:18.031378 23861 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 521687247152450cb8ae19adb7434746 (127.30.153.130:34947): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
W20260810 18:57:18.031495 23856 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:57:18.031622 23856 leader_election.cc:302] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e78028d924414f698e690c07228ebd77; no voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f
I20260810 18:57:18.031808 24497 raft_consensus.cc:2726] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:57:18.065508 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:18.065015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:54109)
0810 18:57:18.065348 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:18.065409 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":93,"threads_started":1}
I20260810 18:57:18.270572 24499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:18.270089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:36437)
0810 18:57:18.270361 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:18.270475 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":94,"threads_started":1}
I20260810 18:57:18.916561 24503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:18.916051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:60827)
0810 18:57:18.916350 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:18.916420 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":88,"threads_started":1}
I20260810 18:57:19.067014 24505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:19.066310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:46915)
0810 18:57:19.066853 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:19.066914 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":358,"threads_started":1}
I20260810 18:57:19.271912 24507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:19.271464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:40677)
0810 18:57:19.271752 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:19.271811 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":81,"threads_started":1}
I20260810 18:57:19.907500 24510 raft_consensus.cc:482] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:19.907675 24510 raft_consensus.cc:504] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:57:19.908217 24510 leader_election.cc:288] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 521687247152450cb8ae19adb7434746 (127.30.153.130:34947), 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
W20260810 18:57:19.909215 23856 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
W20260810 18:57:19.909415 23861 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 521687247152450cb8ae19adb7434746 (127.30.153.130:34947): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:57:19.909508 23861 leader_election.cc:302] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e78028d924414f698e690c07228ebd77; no voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f
I20260810 18:57:19.909757 24510 raft_consensus.cc:2726] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:57:19.917862 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:19.917400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:37339)
0810 18:57:19.917694 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:19.917770 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":98,"threads_started":1}
I20260810 18:57:20.068364 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:20.067855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:55301)
0810 18:57:20.068188 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:20.068266 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":115,"threads_started":1}
I20260810 18:57:20.273319 24513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:20.272744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:57619)
0810 18:57:20.273151 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:20.273219 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":98,"threads_started":1}
I20260810 18:57:20.919152 24515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:20.918701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:38095)
0810 18:57:20.918990 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:20.919050 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":91,"threads_started":1}
I20260810 18:57:21.069669 24516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:21.069159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:47023)
0810 18:57:21.069487 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:21.069568 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":98,"threads_started":1}
I20260810 18:57:21.274645 24517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:21.274118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:36945)
0810 18:57:21.274473 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:21.274544 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":82,"threads_started":1}
I20260810 18:57:21.920564 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:21.920045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:46931)
0810 18:57:21.920338 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:21.920436 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
I20260810 18:57:22.071112 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:22.070638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:47445)
0810 18:57:22.070943 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:22.071007 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":104,"threads_started":1}
I20260810 18:57:22.133471 24524 raft_consensus.cc:482] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:22.133646 24524 raft_consensus.cc:504] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:57:22.134116 24524 leader_election.cc:288] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 521687247152450cb8ae19adb7434746 (127.30.153.130:34947), 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
W20260810 18:57:22.135174 23856 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
W20260810 18:57:22.135298 23861 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 521687247152450cb8ae19adb7434746 (127.30.153.130:34947): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:57:22.135370 23861 leader_election.cc:302] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e78028d924414f698e690c07228ebd77; no voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f
I20260810 18:57:22.135514 24524 raft_consensus.cc:2726] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:57:22.276073 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:22.275612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:45599)
0810 18:57:22.275901 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:22.275969 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":87,"threads_started":1}
I20260810 18:57:22.921972 24526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:22.921504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:60815)
0810 18:57:22.921805 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:22.921871 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":85,"threads_started":1}
I20260810 18:57:23.072429 24527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:23.071974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:58525)
0810 18:57:23.072272 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:23.072333 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":100,"threads_started":1}
I20260810 18:57:23.277534 24528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:23.277068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:41077)
0810 18:57:23.277367 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:23.277431 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":85,"threads_started":1}
I20260810 18:57:23.923666 24530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:23.923111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:45955)
0810 18:57:23.923430 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:23.923502 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":90,"threads_started":1}
I20260810 18:57:24.073920 24532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:24.073426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:49561)
0810 18:57:24.073741 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:24.073814 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":91,"threads_started":1}
I20260810 18:57:24.279134 24533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:24.278558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:43497)
0810 18:57:24.278957 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:24.279028 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":81,"threads_started":1}
I20260810 18:57:24.285622 24534 raft_consensus.cc:482] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:24.285774 24534 raft_consensus.cc:504] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "521687247152450cb8ae19adb7434746" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 34947 } } peers { permanent_uuid: "8ef84dc10a6b414aba7292e25221221f" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 43551 } } peers { permanent_uuid: "e78028d924414f698e690c07228ebd77" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 40599 } }
I20260810 18:57:24.286185 24534 leader_election.cc:288] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 521687247152450cb8ae19adb7434746 (127.30.153.130:34947), 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551)
W20260810 18:57:24.287148 23856 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8ef84dc10a6b414aba7292e25221221f (127.30.153.131:43551): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
W20260810 18:57:24.287266 23861 leader_election.cc:341] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 521687247152450cb8ae19adb7434746 (127.30.153.130:34947): Not found: Tablet not found: 24abec6815b048e59aa61d0bc98ffbd5
I20260810 18:57:24.287365 23861 leader_election.cc:302] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e78028d924414f698e690c07228ebd77; no voters: 521687247152450cb8ae19adb7434746, 8ef84dc10a6b414aba7292e25221221f
I20260810 18:57:24.287523 24534 raft_consensus.cc:2726] T 24abec6815b048e59aa61d0bc98ffbd5 P e78028d924414f698e690c07228ebd77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 18:57:24.925010 24535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:24.924537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:34533)
0810 18:57:24.924846 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:24.924908 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":93,"threads_started":1}
I20260810 18:57:25.075348 24537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:25.074899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:46779)
0810 18:57:25.075198 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:25.075253 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":101,"threads_started":1}
I20260810 18:57:25.280409 24538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:25.279950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:52355)
0810 18:57:25.280244 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:25.280303 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":84,"threads_started":1}
I20260810 18:57:25.926287 24540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:25.925885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:39643)
0810 18:57:25.926130 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:25.926185 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":76,"threads_started":1}
I20260810 18:57:26.076812 24546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:26.076296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:47011)
0810 18:57:26.076627 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:26.076703 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":87,"threads_started":1}
I20260810 18:57:26.281896 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:26.281392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:58615)
0810 18:57:26.281722 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:26.281795 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":109,"threads_started":1}
I20260810 18:57:26.927811 24550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:26.927300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:55237)
0810 18:57:26.927639 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:26.927708 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":117,"threads_started":1}
I20260810 18:57:27.078179 24551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:27.077650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:49681)
0810 18:57:27.077943 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:27.078078 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":88,"threads_started":1}
I20260810 18:57:27.283299 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:27.282800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:58945)
0810 18:57:27.283095 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:27.283195 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":101,"threads_started":1}
I20260810 18:57:27.929329 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:27.928890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.129:41173)
0810 18:57:27.929173 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:27.929233 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":100,"threads_started":1}
I20260810 18:57:28.079604 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:28.079193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.130:51067)
0810 18:57:28.079454 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:28.079508 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
I20260810 18:57:28.270843 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 23763
I20260810 18:57:28.284862 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:28.284384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:37857 (local address 127.30.153.131:36549)
0810 18:57:28.284687 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:28.284750 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.190:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":119,"threads_started":1}
I20260810 18:57:28.286564 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 24051
I20260810 18:57:28.318189 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 24267
2026-08-10T18:57:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59985 ms)
[----------] 30 tests from SecurityITest (257630 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:28 dist-test-slave-68xk krb5kdc[24564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:28 dist-test-slave-68xk krb5kdc[24564](info): set up 2 sockets
Aug 10 18:57:28 dist-test-slave-68xk krb5kdc[24564](info): commencing operation
krb5kdc: starting...
W20260810 18:57:29.593648 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:29 dist-test-slave-68xk krb5kdc[24564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:57:29.729386 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:34055
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:34055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:29.842301 24588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:29.842648 24588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:29.842742 24588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:29.847080 24588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:29.847172 24588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:29.847213 24588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:29.847249 24588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:29.847282 24588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:29.852438 24588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:34055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:34055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:29.853467 24588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:29.854511 24588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:29.863569 24595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:29.863785 24588 server_base.cc:1034] running on GCE node
W20260810 18:57:29.863610 24596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:29.863627 24598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:29.864539 24588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:29.865739 24588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:29.866914 24588 hybrid_clock.cc:648] HybridClock initialized: now 1786388249866884 us; error 41 us; skew 500 ppm
Aug 10 18:57:29 dist-test-slave-68xk krb5kdc[24564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388249, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:29.874212 24588 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:57:29.875182 24588 webserver.cc:460] Webserver started at http://127.30.153.190:44175/ using document root <none> and password file <none>
I20260810 18:57:29.875742 24588 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:29.875804 24588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:29.875967 24588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:29.878003 24588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "805c3e91727e43fa928570925489bd97"
format_stamp: "Formatted at 2026-08-10 18:57:29 on dist-test-slave-68xk"
server_key: "5200d258edfc57da0278a8603e7a22f7"
server_key_iv: "be7b7d281a78948985728f75fc3c02db"
server_key_version: "encryptionkey@0"
I20260810 18:57:29.878625 24588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "805c3e91727e43fa928570925489bd97"
format_stamp: "Formatted at 2026-08-10 18:57:29 on dist-test-slave-68xk"
server_key: "5200d258edfc57da0278a8603e7a22f7"
server_key_iv: "be7b7d281a78948985728f75fc3c02db"
server_key_version: "encryptionkey@0"
I20260810 18:57:29.882704 24588 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:57:29.885291 24607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:29.886814 24588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:29.886962 24588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "805c3e91727e43fa928570925489bd97"
format_stamp: "Formatted at 2026-08-10 18:57:29 on dist-test-slave-68xk"
server_key: "5200d258edfc57da0278a8603e7a22f7"
server_key_iv: "be7b7d281a78948985728f75fc3c02db"
server_key_version: "encryptionkey@0"
I20260810 18:57:29.887104 24588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:29.897504 24588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:29.900460 24588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:29.900664 24588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:29.908345 24588 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:34055
I20260810 18:57:29.908370 24696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:34055 every 8 connection(s)
I20260810 18:57:29.909314 24588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:29.912147 24698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:29.916257 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 24588
I20260810 18:57:29.916402 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:29.916702 31334 external_mini_cluster.cc:1428] Setting key 782af872c7d67df02852824a145008dd
I20260810 18:57:29.918239 24698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97: Bootstrap starting.
I20260810 18:57:29.920512 24698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:29.921269 24698 log.cc:824] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:29.923184 24698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97: No bootstrap required, opened a new log
Aug 10 18:57:29 dist-test-slave-68xk krb5kdc[24564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:29.927012 24698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805c3e91727e43fa928570925489bd97" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34055 } }
I20260810 18:57:29.927268 24698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:29.927325 24698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805c3e91727e43fa928570925489bd97, State: Initialized, Role: FOLLOWER
I20260810 18:57:29.927840 24698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805c3e91727e43fa928570925489bd97" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34055 } }
I20260810 18:57:29.928010 24698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:29.928092 24698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:29.928220 24698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:29.929072 24698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805c3e91727e43fa928570925489bd97" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34055 } }
I20260810 18:57:29.929481 24698 leader_election.cc:302] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 805c3e91727e43fa928570925489bd97; no voters:
I20260810 18:57:29.929848 24698 leader_election.cc:288] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:29.929962 24707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:29.930222 24707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [term 1 LEADER]: Becoming Leader. State: Replica: 805c3e91727e43fa928570925489bd97, State: Running, Role: LEADER
I20260810 18:57:29.930646 24707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805c3e91727e43fa928570925489bd97" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34055 } }
I20260810 18:57:29.931255 24698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:29.931988 24707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 805c3e91727e43fa928570925489bd97. Latest consensus state: current_term: 1 leader_uuid: "805c3e91727e43fa928570925489bd97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805c3e91727e43fa928570925489bd97" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34055 } } }
I20260810 18:57:29.932206 24707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:29.932443 24708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "805c3e91727e43fa928570925489bd97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805c3e91727e43fa928570925489bd97" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34055 } } }
I20260810 18:57:29.932561 24708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:29.932687 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:29.918177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.30.153.190:34055)
0810 18:57:29.918668 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:29.918686 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:29.918721 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:57:29.919382 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:29.919397 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:29.919715 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:29.920051 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:29.920922 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:29.922015 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:29.922965 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:29.923501 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:29.923674 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:29.926300 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:29.926305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:29.926318 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:29.926345 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:29.928438 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:29.929018 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:29.929026 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:29.929031 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:29.929169 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:29.929534 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:29.929537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:29.929538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:29.930113 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:29.930392 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:29.931228 (+ 836us) server_negotiation.cc:299] Negotiation successful
0810 18:57:29.931557 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":174,"threads_started":1}
W20260810 18:57:29.936494 24731 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:57:29.936602 24731 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:57:29.936681 24719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:29.937402 24719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:29.943424 24719 catalog_manager.cc:1347] Generated new cluster ID: ad07bc73022046978583b21c520615f5
I20260810 18:57:29.943495 24719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:29.967945 24719 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:29.969331 24719 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:29.976370 24719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97: Generated new TSK 0
I20260810 18:57:29.977145 24719 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:57:30.021293 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34055
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:30.140714 24740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:30.140959 24740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:30.141000 24740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:30.144884 24740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:30.144936 24740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:30.144990 24740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:57:30.150332 24740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:30.151450 24740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:30.152498 24740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:30.162402 24751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:30.162379 24748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:30.162377 24753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:30.162567 24740 server_base.cc:1034] running on GCE node
I20260810 18:57:30.163141 24740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:30.163806 24740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:30.164991 24740 hybrid_clock.cc:648] HybridClock initialized: now 1786388250164971 us; error 33 us; skew 500 ppm
Aug 10 18:57:30 dist-test-slave-68xk krb5kdc[24564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388250, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:30.167928 24740 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:57:30.168830 24740 webserver.cc:460] Webserver started at http://127.30.153.129:38053/ using document root <none> and password file <none>
I20260810 18:57:30.169359 24740 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:30.169420 24740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:30.169595 24740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:30.171669 24740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "84b2b3fde38b4aecaab7e33ed9f24192"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "2bc73ffbf23529fb282ca7ad185b0583"
server_key_iv: "b7db028016d7340da393ddfa28fceb6c"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.172192 24740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "84b2b3fde38b4aecaab7e33ed9f24192"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "2bc73ffbf23529fb282ca7ad185b0583"
server_key_iv: "b7db028016d7340da393ddfa28fceb6c"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.175971 24740 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:30.178475 24766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.179807 24740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:57:30.179960 24740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "84b2b3fde38b4aecaab7e33ed9f24192"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "2bc73ffbf23529fb282ca7ad185b0583"
server_key_iv: "b7db028016d7340da393ddfa28fceb6c"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.180058 24740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:30.193006 24740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:30.195854 24740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:30.196058 24740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:30.196655 24740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:30.197463 24740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:30.197535 24740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.197651 24740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:30.197707 24740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.206974 24740 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:46011
I20260810 18:57:30.207015 24946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:46011 every 8 connection(s)
I20260810 18:57:30.207880 24740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:57:30.209084 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 24740
I20260810 18:57:30.209196 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:57:30.209483 31334 external_mini_cluster.cc:1428] Setting key 01ed15d1d81f03d102068d8732712fa9
Aug 10 18:57:30 dist-test-slave-68xk krb5kdc[24564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388250, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:57:30.223366 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.209829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:33139 (local address 127.30.153.190:34055)
0810 18:57:30.209971 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.209976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.210892 (+ 916us) server_negotiation.cc:407] Connection header received
0810 18:57:30.212297 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.212303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.212365 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.212466 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:30.213945 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.214554 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.216359 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.216640 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.216754 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.219481 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:30.219485 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:30.219490 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:30.219523 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:30.221291 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:30.221861 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:30.221864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:30.221865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:30.221948 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:30.222314 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:30.222317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:30.222318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:30.222581 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:30.222663 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.223124 (+ 461us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.223236 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
I20260810 18:57:30.224334 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.210200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:34055 (local address 127.30.153.129:33139)
0810 18:57:30.210734 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:30.210791 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:30.212059 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:30.212620 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:30.212627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:30.212960 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:30.213767 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.213792 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.214680 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.214686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.216207 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.216223 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.216832 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.216838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.217073 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.217769 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:30.217785 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:30.219297 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:30.221411 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:30.221415 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:30.221427 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:30.221758 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:30.222068 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:30.222071 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:30.222072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:30.222217 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:30.222688 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:30.222694 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:30.222917 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 18:57:30.223077 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:57:30.223328 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":163,"threads_started":1}
I20260810 18:57:30.225703 24948 heartbeater.cc:347] Connected to a master server at 127.30.153.190:34055
I20260810 18:57:30.225941 24948 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:30.226513 24948 heartbeater.cc:511] Master 127.30.153.190:34055 requested a full tablet report, sending...
I20260810 18:57:30.228287 24631 ts_manager.cc:194] Registered new tserver with Master: 84b2b3fde38b4aecaab7e33ed9f24192 (127.30.153.129:46011)
I20260810 18:57:30.229995 24631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:33139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:57:30.241806 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34055
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:30.353971 24954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:30.354290 24954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:30.354378 24954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:30.358300 24954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:30.358386 24954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:30.358510 24954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:57:30.363785 24954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:30.364837 24954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:30.365901 24954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:30.375834 24971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:30.375824 24970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:30.375882 24975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:30.376032 24954 server_base.cc:1034] running on GCE node
I20260810 18:57:30.376520 24954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:30.377177 24954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:30.378373 24954 hybrid_clock.cc:648] HybridClock initialized: now 1786388250378342 us; error 50 us; skew 500 ppm
Aug 10 18:57:30 dist-test-slave-68xk krb5kdc[24564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388250, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:30.381124 24954 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:57:30.381982 24954 webserver.cc:460] Webserver started at http://127.30.153.130:36629/ using document root <none> and password file <none>
I20260810 18:57:30.382637 24954 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:30.382731 24954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:30.382989 24954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:30.385262 24954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "bdc8c4c64be64b769434068424d73512"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "19579696b95e2da40e0e454cb7c9450f"
server_key_iv: "e967afcb946ee47175079e8ef1c61e6e"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.385842 24954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "bdc8c4c64be64b769434068424d73512"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "19579696b95e2da40e0e454cb7c9450f"
server_key_iv: "e967afcb946ee47175079e8ef1c61e6e"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.389636 24954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:57:30.392307 24988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.393491 24954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:57:30.393639 24954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "bdc8c4c64be64b769434068424d73512"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "19579696b95e2da40e0e454cb7c9450f"
server_key_iv: "e967afcb946ee47175079e8ef1c61e6e"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.393757 24954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:30.415043 24954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:30.417836 24954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:30.418097 24954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:30.418769 24954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:30.419636 24954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:30.419711 24954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.419782 24954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:30.419828 24954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.428977 24954 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:44213
I20260810 18:57:30.429050 25172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:44213 every 8 connection(s)
I20260810 18:57:30.429872 24954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:57:30.438591 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 24954
I20260810 18:57:30.438740 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:57:30.439029 31334 external_mini_cluster.cc:1428] Setting key 337dbcbc9374078e24246f669de36f25
Aug 10 18:57:30 dist-test-slave-68xk krb5kdc[24564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388250, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:30.444185 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.431735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:57657 (local address 127.30.153.190:34055)
0810 18:57:30.431874 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.431877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.432948 (+ 1071us) server_negotiation.cc:407] Connection header received
0810 18:57:30.434304 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.434307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.434366 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.434509 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:30.435628 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.436141 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.437256 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.437469 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.437539 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.440172 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:30.440175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:30.440177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:30.440201 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:30.441741 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:30.442380 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:30.442383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:30.442384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:30.442545 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:30.442946 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:30.442952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:30.442954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:30.443235 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:30.443349 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.443802 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.443941 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:57:30.444780 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.432236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:34055 (local address 127.30.153.130:57657)
0810 18:57:30.432772 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:30.432833 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:30.434067 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:30.434661 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:30.434667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:30.435010 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:30.435494 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.435507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.436280 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.436282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.437137 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.437146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.437590 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.437592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.437726 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.438262 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:30.438277 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:30.440021 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:30.441862 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:30.441871 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:30.441886 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:30.442277 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:30.442665 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:30.442671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:30.442675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:30.442844 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:30.443347 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:30.443352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:30.443564 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 18:57:30.443714 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 18:57:30.443946 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":106,"threads_started":1}
I20260810 18:57:30.446060 25174 heartbeater.cc:347] Connected to a master server at 127.30.153.190:34055
I20260810 18:57:30.446328 25174 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:30.446950 25174 heartbeater.cc:511] Master 127.30.153.190:34055 requested a full tablet report, sending...
I20260810 18:57:30.448134 24631 ts_manager.cc:194] Registered new tserver with Master: bdc8c4c64be64b769434068424d73512 (127.30.153.130:44213)
I20260810 18:57:30.448716 24631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:57657
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:57:30.469138 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34055
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:30.593385 25180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:30.593667 25180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:30.593712 25180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:30.597828 25180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:30.597894 25180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:30.597954 25180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:57:30.604030 25180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:34055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:30.605135 25180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:30.606148 25180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:30.615970 25193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:30.615990 25186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:30.616183 25180 server_base.cc:1034] running on GCE node
W20260810 18:57:30.616050 25188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:30.616751 25180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:30.617468 25180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:30.618666 25180 hybrid_clock.cc:648] HybridClock initialized: now 1786388250618618 us; error 60 us; skew 500 ppm
Aug 10 18:57:30 dist-test-slave-68xk krb5kdc[24564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388250, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:30.621567 25180 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:57:30.622563 25180 webserver.cc:460] Webserver started at http://127.30.153.131:42847/ using document root <none> and password file <none>
I20260810 18:57:30.623152 25180 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:30.623256 25180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:30.623497 25180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:30.625623 25180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "cedf5fac2b9048f693c85c33b998f634"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "e91748f1abe22748db9f2e71d61e3b6a"
server_key_iv: "b9d3a5564bc803aba741d3f38e91d67f"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.626235 25180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "cedf5fac2b9048f693c85c33b998f634"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "e91748f1abe22748db9f2e71d61e3b6a"
server_key_iv: "b9d3a5564bc803aba741d3f38e91d67f"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.629951 25180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:57:30.632618 25205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.633872 25180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:30.634016 25180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "cedf5fac2b9048f693c85c33b998f634"
format_stamp: "Formatted at 2026-08-10 18:57:30 on dist-test-slave-68xk"
server_key: "e91748f1abe22748db9f2e71d61e3b6a"
server_key_iv: "b9d3a5564bc803aba741d3f38e91d67f"
server_key_version: "encryptionkey@0"
I20260810 18:57:30.634130 25180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:30.654045 25180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:30.657204 25180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:30.657433 25180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:30.658048 25180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:30.658996 25180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:30.659075 25180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.659160 25180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:30.659219 25180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:30.668654 25180 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:44361
I20260810 18:57:30.668684 25399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:44361 every 8 connection(s)
I20260810 18:57:30.669553 25180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:57:30.676436 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 25180
I20260810 18:57:30.676599 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:57:30.676978 31334 external_mini_cluster.cc:1428] Setting key c33d62db81c80d62f1b5045bfc341140
Aug 10 18:57:30 dist-test-slave-68xk krb5kdc[24564](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388250, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:30.684180 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.671365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:60129 (local address 127.30.153.190:34055)
0810 18:57:30.671550 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.671555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.672401 (+ 846us) server_negotiation.cc:407] Connection header received
0810 18:57:30.673660 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.673662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.673711 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.673792 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:30.675011 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.675572 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.676820 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.677151 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.677280 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.680470 (+ 3190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:30.680473 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:30.680475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:30.680501 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:30.681915 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:30.682540 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:30.682543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:30.682544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:30.682618 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:30.682993 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:30.682995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:30.682996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:30.683198 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:30.683312 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.683905 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.684018 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 18:57:30.685062 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.671772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:34055 (local address 127.30.153.131:60129)
0810 18:57:30.672241 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:30.672286 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:30.673461 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:30.673969 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:30.673976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:30.674360 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:30.674873 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.674887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.675686 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.675689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.676675 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.676688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.677343 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.677350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.677574 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.678322 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:30.678350 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:30.680294 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:30.682033 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:30.682038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:30.682057 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:30.682365 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:30.682729 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:30.682731 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:30.682733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:30.682893 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:30.683326 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:30.683331 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:30.683575 (+ 244us) client_negotiation.cc:769] Sending connection context
0810 18:57:30.683741 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 18:57:30.684089 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":107,"threads_started":1}
I20260810 18:57:30.686326 25400 heartbeater.cc:347] Connected to a master server at 127.30.153.190:34055
I20260810 18:57:30.686628 25400 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:30.687170 25400 heartbeater.cc:511] Master 127.30.153.190:34055 requested a full tablet report, sending...
I20260810 18:57:30.688278 24631 ts_manager.cc:194] Registered new tserver with Master: cedf5fac2b9048f693c85c33b998f634 (127.30.153.131:44361)
I20260810 18:57:30.688874 24631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:60129
I20260810 18:57:30.691887 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:57:30.701596 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.694034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.30.153.190:34055)
0810 18:57:30.694165 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.694169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.694328 (+ 159us) server_negotiation.cc:407] Connection header received
0810 18:57:30.694463 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.694468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.694521 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.694589 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:30.695358 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.695859 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.696730 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.696963 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.697041 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.698602 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:30.698605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:30.698607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:30.698628 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:30.700097 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:30.700486 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:30.700488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:30.700489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:30.700562 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:30.700855 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:30.700858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:30.700859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:30.701060 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:30.701163 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.701386 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.701480 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:57:30.704628 24631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:57:30.707796 24631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:57:30.722940 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.717008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44390 (local address 127.30.153.131:44361)
0810 18:57:30.717815 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.717832 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.717857 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:57:30.717934 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.717941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.718128 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.718311 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:30.718611 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.719726 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.721241 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.722074 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.722207 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.722348 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.722674 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.722788 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":693,"thread_start_us":165,"threads_started":1}
I20260810 18:57:30.722963 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.716339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:44361 (local address 127.0.0.1:44390)
0810 18:57:30.717167 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:30.717182 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:30.717295 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:30.718296 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:30.718298 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:30.718311 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:30.718488 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.718494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.719929 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.719934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.721082 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.721092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.722207 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.722211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.722438 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.722510 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 18:57:30.722626 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 18:57:30.722768 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":163,"thread_start_us":100,"threads_started":1}
I20260810 18:57:30.722965 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.716612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46011 (local address 127.0.0.1:55310)
0810 18:57:30.717532 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:30.717544 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:30.717627 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:30.718256 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:30.718261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:30.718285 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:30.718481 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.718488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.719531 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.719537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.721066 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.721073 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.722207 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.722211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.722406 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.722515 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 18:57:30.722626 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 18:57:30.722768 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":226,"thread_start_us":113,"threads_started":1}
I20260810 18:57:30.724057 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.716261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:44213 (local address 127.0.0.1:59526)
0810 18:57:30.717093 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:30.717145 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:30.717295 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:30.718047 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:30.718053 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:30.718078 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:30.718225 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.718237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.719822 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.719825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.721049 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.721061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.723654 (+ 2593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.723657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.723748 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.723770 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:57:30.723843 (+ 73us) client_negotiation.cc:240] Negotiation successful
0810 18:57:30.723915 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":60,"thread_start_us":96,"threads_started":1}
I20260810 18:57:30.724320 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.717096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.30.153.129:46011)
0810 18:57:30.717549 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.717567 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.717608 (+ 41us) server_negotiation.cc:407] Connection header received
0810 18:57:30.717784 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.717791 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.717963 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.718102 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:30.718643 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.719396 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.721186 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.722074 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.722216 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.722348 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.724072 (+ 1724us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.724190 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":103,"threads_started":1}
I20260810 18:57:30.724481 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.716436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.30.153.130:44213)
0810 18:57:30.717068 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.717080 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.717440 (+ 360us) server_negotiation.cc:407] Connection header received
0810 18:57:30.717529 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.717534 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.717780 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.717926 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:30.718370 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.719659 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.722465 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.723348 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.723470 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.723589 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.724237 (+ 648us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.724332 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":126,"threads_started":1}
I20260810 18:57:30.726033 25285 tablet_service.cc:1467] Processing CreateTablet for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e (DEFAULT_TABLE table=test-table [id=a2c80a91d67c4f27afd8ab90adf5b32e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:30.726715 24839 tablet_service.cc:1467] Processing CreateTablet for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e (DEFAULT_TABLE table=test-table [id=a2c80a91d67c4f27afd8ab90adf5b32e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:30.727175 25285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:30.727617 24839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:30.728961 25063 tablet_service.cc:1467] Processing CreateTablet for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e (DEFAULT_TABLE table=test-table [id=a2c80a91d67c4f27afd8ab90adf5b32e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:30.729954 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:30.733922 25424 tablet_bootstrap.cc:492] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192: Bootstrap starting.
I20260810 18:57:30.736907 25424 tablet_bootstrap.cc:654] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:30.736878 25431 tablet_bootstrap.cc:492] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Bootstrap starting.
I20260810 18:57:30.737442 25427 tablet_bootstrap.cc:492] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Bootstrap starting.
I20260810 18:57:30.737922 25424 log.cc:824] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:30.739053 25431 tablet_bootstrap.cc:654] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:30.739549 25427 tablet_bootstrap.cc:654] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:30.740376 25431 log.cc:824] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:30.740554 25424 tablet_bootstrap.cc:492] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192: No bootstrap required, opened a new log
I20260810 18:57:30.740808 25424 ts_tablet_manager.cc:1397] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 18:57:30.740782 25427 log.cc:824] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:30.742074 25431 tablet_bootstrap.cc:492] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: No bootstrap required, opened a new log
I20260810 18:57:30.742265 25431 ts_tablet_manager.cc:1397] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:57:30.744084 25427 tablet_bootstrap.cc:492] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: No bootstrap required, opened a new log
I20260810 18:57:30.744277 25427 ts_tablet_manager.cc:1397] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260810 18:57:30.745016 25424 raft_consensus.cc:348] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.745322 25424 raft_consensus.cc:374] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:30.745389 25424 raft_consensus.cc:729] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b2b3fde38b4aecaab7e33ed9f24192, State: Initialized, Role: FOLLOWER
I20260810 18:57:30.745561 25431 raft_consensus.cc:348] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.745829 25431 raft_consensus.cc:374] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:30.745893 25431 raft_consensus.cc:729] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc8c4c64be64b769434068424d73512, State: Initialized, Role: FOLLOWER
I20260810 18:57:30.745844 25424 consensus_queue.cc:260] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.746322 25431 consensus_queue.cc:260] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.746917 25424 ts_tablet_manager.cc:1428] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:57:30.747013 25174 heartbeater.cc:503] Master 127.30.153.190:34055 was elected leader, sending a full tablet report...
I20260810 18:57:30.747249 24948 heartbeater.cc:503] Master 127.30.153.190:34055 was elected leader, sending a full tablet report...
I20260810 18:57:30.747887 25427 raft_consensus.cc:348] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.748126 25427 raft_consensus.cc:374] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:30.748194 25427 raft_consensus.cc:729] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cedf5fac2b9048f693c85c33b998f634, State: Initialized, Role: FOLLOWER
I20260810 18:57:30.748636 25427 consensus_queue.cc:260] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.749248 25400 heartbeater.cc:503] Master 127.30.153.190:34055 was elected leader, sending a full tablet report...
I20260810 18:57:30.749336 25431 ts_tablet_manager.cc:1428] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260810 18:57:30.749569 25427 ts_tablet_manager.cc:1428] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260810 18:57:30.909343 25441 raft_consensus.cc:482] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:30.909564 25441 raft_consensus.cc:504] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.910945 25441 leader_election.cc:288] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cedf5fac2b9048f693c85c33b998f634 (127.30.153.131:44361), 84b2b3fde38b4aecaab7e33ed9f24192 (127.30.153.129:46011)
I20260810 18:57:30.915577 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.911171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:44361 (local address 127.30.153.130:37047)
0810 18:57:30.911286 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:30.911302 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:30.911394 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:30.912034 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:30.912037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:30.912053 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:30.912176 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.912181 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.912991 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.912994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.914181 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.914191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.915113 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.915120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.915237 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.915265 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:57:30.915367 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:57:30.915446 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 18:57:30.915577 25421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.911640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:37047 (local address 127.30.153.131:44361)
0810 18:57:30.911785 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.911789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.911806 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:57:30.911863 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.911866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.911917 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.912005 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:30.912317 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.912865 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.914332 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.914970 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.915089 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.915141 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.915390 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.915468 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:57:30.915691 25423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.911357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:59095 (local address 127.30.153.129:46011)
0810 18:57:30.911482 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.911487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.911786 (+ 299us) server_negotiation.cc:407] Connection header received
0810 18:57:30.911881 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.911884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.911934 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.912009 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:30.912467 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.913201 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.914612 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.915062 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.915180 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.915235 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.915479 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.915570 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:57:30.916181 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.911182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46011 (local address 127.30.153.130:59095)
0810 18:57:30.911664 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:30.911678 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:30.911795 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:30.912149 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:30.912152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:30.912170 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:30.912319 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.912326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.913336 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.913340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.914486 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:30.914493 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.915228 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.915232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:30.915336 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.915359 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:57:30.915976 (+ 617us) client_negotiation.cc:240] Negotiation successful
0810 18:57:30.916052 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":108,"thread_start_us":118,"threads_started":1}
I20260810 18:57:30.916524 25321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48fa3cf5e77e47d6a73b0fe0f21d072e" candidate_uuid: "bdc8c4c64be64b769434068424d73512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cedf5fac2b9048f693c85c33b998f634" is_pre_election: true
I20260810 18:57:30.916857 25321 raft_consensus.cc:2454] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdc8c4c64be64b769434068424d73512 in term 0.
I20260810 18:57:30.916857 24869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48fa3cf5e77e47d6a73b0fe0f21d072e" candidate_uuid: "bdc8c4c64be64b769434068424d73512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" is_pre_election: true
I20260810 18:57:30.917164 24869 raft_consensus.cc:2454] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdc8c4c64be64b769434068424d73512 in term 0.
I20260810 18:57:30.917378 24990 leader_election.cc:302] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdc8c4c64be64b769434068424d73512, cedf5fac2b9048f693c85c33b998f634; no voters:
I20260810 18:57:30.917686 25441 raft_consensus.cc:2781] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:57:30.917778 25441 raft_consensus.cc:482] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:57:30.917822 25441 raft_consensus.cc:3037] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:30.919001 25441 raft_consensus.cc:504] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.919404 25441 leader_election.cc:288] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [CANDIDATE]: Term 1 election: Requested vote from peers cedf5fac2b9048f693c85c33b998f634 (127.30.153.131:44361), 84b2b3fde38b4aecaab7e33ed9f24192 (127.30.153.129:46011)
I20260810 18:57:30.919754 24869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48fa3cf5e77e47d6a73b0fe0f21d072e" candidate_uuid: "bdc8c4c64be64b769434068424d73512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b2b3fde38b4aecaab7e33ed9f24192"
I20260810 18:57:30.919878 24869 raft_consensus.cc:3037] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192 [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:57:30.920466 25401 tablet.cc:2369] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:57:30.921204 24869 raft_consensus.cc:2454] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdc8c4c64be64b769434068424d73512 in term 1.
I20260810 18:57:30.921461 25321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48fa3cf5e77e47d6a73b0fe0f21d072e" candidate_uuid: "bdc8c4c64be64b769434068424d73512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cedf5fac2b9048f693c85c33b998f634"
I20260810 18:57:30.921568 25321 raft_consensus.cc:3037] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:30.921576 24989 leader_election.cc:302] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84b2b3fde38b4aecaab7e33ed9f24192, bdc8c4c64be64b769434068424d73512; no voters:
I20260810 18:57:30.921747 25441 raft_consensus.cc:2781] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:30.921988 25441 raft_consensus.cc:686] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 1 LEADER]: Becoming Leader. State: Replica: bdc8c4c64be64b769434068424d73512, State: Running, Role: LEADER
I20260810 18:57:30.922330 25441 consensus_queue.cc:237] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } }
I20260810 18:57:30.922829 25321 raft_consensus.cc:2454] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdc8c4c64be64b769434068424d73512 in term 1.
I20260810 18:57:30.925596 24630 catalog_manager.cc:5495] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdc8c4c64be64b769434068424d73512 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "bdc8c4c64be64b769434068424d73512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc8c4c64be64b769434068424d73512" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 } health_report { overall_health: UNKNOWN } } }
W20260810 18:57:30.931089 25176 tablet.cc:2369] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:57:30.959304 24950 tablet.cc:2369] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:57:30.964581 25419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:30.960878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59534 (local address 127.30.153.130:44213)
0810 18:57:30.961017 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:30.961021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:30.961035 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:57:30.961184 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:30.961186 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:30.961246 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:30.961336 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:57:30.961682 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.962206 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.963108 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:30.963333 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:30.963417 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:30.963761 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:57:30.964169 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:57:30.964281 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:30.964348 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 18:57:30.964403 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 18:57:30.970665 25321 raft_consensus.cc:1261] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 1 FOLLOWER]: Refusing update from remote peer bdc8c4c64be64b769434068424d73512: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:30.970664 24869 raft_consensus.cc:1261] T 48fa3cf5e77e47d6a73b0fe0f21d072e P 84b2b3fde38b4aecaab7e33ed9f24192 [term 1 FOLLOWER]: Refusing update from remote peer bdc8c4c64be64b769434068424d73512: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:30.971385 25441 consensus_queue.cc:1035] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b2b3fde38b4aecaab7e33ed9f24192" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 46011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:30.971621 25446 consensus_queue.cc:1035] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cedf5fac2b9048f693c85c33b998f634" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:30.980085 25455 mvcc.cc:205] Tried to move back new op lower bound from 7317046275968217088 to 7317046275781025792. Current Snapshot: MvccSnapshot[applied={T|T < 7317046275968217088}]
I20260810 18:57:30.980285 25454 mvcc.cc:205] Tried to move back new op lower bound from 7317046275968217088 to 7317046275781025792. Current Snapshot: MvccSnapshot[applied={T|T < 7317046275968217088}]
I20260810 18:57:30.988337 24631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:30.988498 24631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:30.991274 24631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 805c3e91727e43fa928570925489bd97: Sending DeleteTablet for 3 replicas of tablet 48fa3cf5e77e47d6a73b0fe0f21d072e
I20260810 18:57:30.992089 25063 tablet_service.cc:1514] Processing DeleteTablet for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:30 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:59526
I20260810 18:57:30.992192 25285 tablet_service.cc:1514] Processing DeleteTablet for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:30 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:44390
I20260810 18:57:30.992609 24839 tablet_service.cc:1514] Processing DeleteTablet for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:30 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:55310
I20260810 18:57:30.993014 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 24740
I20260810 18:57:30.999639 25466 tablet_replica.cc:330] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: stopping tablet replica
I20260810 18:57:30.999974 25466 raft_consensus.cc:2229] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:57:31.000353 25466 raft_consensus.cc:2258] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:31.001848 25466 ts_tablet_manager.cc:1905] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:31.004458 25466 ts_tablet_manager.cc:1918] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:31.004573 25466 log.cc:1196] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/48fa3cf5e77e47d6a73b0fe0f21d072e
I20260810 18:57:31.004940 25466 ts_tablet_manager.cc:1939] T 48fa3cf5e77e47d6a73b0fe0f21d072e P bdc8c4c64be64b769434068424d73512: Deleting consensus metadata
I20260810 18:57:31.006573 24611 catalog_manager.cc:4841] TS bdc8c4c64be64b769434068424d73512 (127.30.153.130:44213): tablet 48fa3cf5e77e47d6a73b0fe0f21d072e (table test-table [id=a2c80a91d67c4f27afd8ab90adf5b32e]) successfully deleted
I20260810 18:57:31.010672 25467 tablet_replica.cc:330] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: stopping tablet replica
I20260810 18:57:31.011155 25467 raft_consensus.cc:2229] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:31.011586 25467 raft_consensus.cc:2258] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 18:57:31.012341 24609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:46011 (error 108)
I20260810 18:57:31.013441 25416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.013104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:46011 (local address 127.0.0.1:55324)
0810 18:57:31.013250 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:31.013334 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:46011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 18:57:31.013582 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 24954
I20260810 18:57:31.013665 25467 ts_tablet_manager.cc:1905] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 18:57:31.013764 24609 catalog_manager.cc:4584] TS 84b2b3fde38b4aecaab7e33ed9f24192 (127.30.153.129:46011): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48fa3cf5e77e47d6a73b0fe0f21d072e: Network error: Client connection negotiation failed: client connection to 127.30.153.129:46011: connect: Connection refused (error 111)
I20260810 18:57:31.017942 25467 ts_tablet_manager.cc:1918] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:31.018028 25467 log.cc:1196] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/48fa3cf5e77e47d6a73b0fe0f21d072e
I20260810 18:57:31.018358 25467 ts_tablet_manager.cc:1939] T 48fa3cf5e77e47d6a73b0fe0f21d072e P cedf5fac2b9048f693c85c33b998f634: Deleting consensus metadata
I20260810 18:57:31.019585 24610 catalog_manager.cc:4841] TS cedf5fac2b9048f693c85c33b998f634 (127.30.153.131:44361): tablet 48fa3cf5e77e47d6a73b0fe0f21d072e (table test-table [id=a2c80a91d67c4f27afd8ab90adf5b32e]) successfully deleted
I20260810 18:57:31.023561 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 25180
I20260810 18:57:31.036736 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 24588
2026-08-10T18:57:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2719 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T18:57:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:31Z Disabled control of system clock
I20260810 18:57:31.077353 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:39197
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:39197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:57:31.188421 25477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:31.188696 25477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:31.188732 25477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:31.192610 25477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:31.192687 25477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:31.192703 25477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:31.192719 25477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:31.192734 25477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:31.197403 25477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:39197
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:39197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:31.198431 25477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:31.199738 25477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:31.208386 25493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:31.208439 25489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:31.208386 25488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:31.208616 25477 server_base.cc:1034] running on GCE node
I20260810 18:57:31.209235 25477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:31.210291 25477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:31.211464 25477 hybrid_clock.cc:648] HybridClock initialized: now 1786388251211441 us; error 37 us; skew 500 ppm
I20260810 18:57:31.213258 25477 webserver.cc:460] Webserver started at http://127.30.153.190:40115/ using document root <none> and password file <none>
I20260810 18:57:31.213794 25477 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:31.213853 25477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:31.214018 25477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:31.216063 25477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "00af0e9de39246419d585ef6f64f0620"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "39fafac4b0e07b50b47038776ed5e658"
server_key_iv: "486b362f024d8e533a91c9e38a17dbc6"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.216578 25477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "00af0e9de39246419d585ef6f64f0620"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "39fafac4b0e07b50b47038776ed5e658"
server_key_iv: "486b362f024d8e533a91c9e38a17dbc6"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.220217 25477 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260810 18:57:31.222587 25505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.223817 25477 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:31.223942 25477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "00af0e9de39246419d585ef6f64f0620"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "39fafac4b0e07b50b47038776ed5e658"
server_key_iv: "486b362f024d8e533a91c9e38a17dbc6"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.224048 25477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:31.237897 25477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:31.238583 25477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:31.238785 25477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:31.245966 25477 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:39197
I20260810 18:57:31.245975 25578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:39197 every 8 connection(s)
I20260810 18:57:31.246909 25477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:31.249981 25580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:31.253207 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 25477
I20260810 18:57:31.253377 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:31.253649 31334 external_mini_cluster.cc:1428] Setting key 13d0d0ee9aca517a9e5a125d44ffcc72
I20260810 18:57:31.255880 25580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620: Bootstrap starting.
I20260810 18:57:31.258285 25580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:31.259295 25580 log.cc:824] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:31.261529 25580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620: No bootstrap required, opened a new log
I20260810 18:57:31.263157 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.255124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.30.153.190:39197)
0810 18:57:31.255774 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:31.255790 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:31.255825 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:57:31.256483 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:31.256502 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:31.256897 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:31.257171 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:31.257520 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.258544 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.259802 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.260395 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.260518 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.261384 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:31.261389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:31.261401 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:31.261839 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:31.261891 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:31.261912 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:31.261987 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:31.262154 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 18:57:31.262452 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":135,"threads_started":1}
I20260810 18:57:31.265120 25580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00af0e9de39246419d585ef6f64f0620" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39197 } }
I20260810 18:57:31.265395 25580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:31.265445 25580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00af0e9de39246419d585ef6f64f0620, State: Initialized, Role: FOLLOWER
I20260810 18:57:31.265904 25580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00af0e9de39246419d585ef6f64f0620" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39197 } }
I20260810 18:57:31.266062 25580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:31.266111 25580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:31.266183 25580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:31.267105 25580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00af0e9de39246419d585ef6f64f0620" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39197 } }
I20260810 18:57:31.267388 25580 leader_election.cc:302] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00af0e9de39246419d585ef6f64f0620; no voters:
I20260810 18:57:31.267627 25580 leader_election.cc:288] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:31.267745 25587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:31.268031 25587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [term 1 LEADER]: Becoming Leader. State: Replica: 00af0e9de39246419d585ef6f64f0620, State: Running, Role: LEADER
I20260810 18:57:31.268378 25587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00af0e9de39246419d585ef6f64f0620" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39197 } }
I20260810 18:57:31.268880 25580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:31.270157 25592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00af0e9de39246419d585ef6f64f0620. Latest consensus state: current_term: 1 leader_uuid: "00af0e9de39246419d585ef6f64f0620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00af0e9de39246419d585ef6f64f0620" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39197 } } }
I20260810 18:57:31.270318 25592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:31.270135 25589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00af0e9de39246419d585ef6f64f0620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00af0e9de39246419d585ef6f64f0620" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 39197 } } }
I20260810 18:57:31.270592 25589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:31.270746 25605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:31.273684 25605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:31.279137 25605 catalog_manager.cc:1347] Generated new cluster ID: c86559098dff4a7da6832dabd0e3f4e4
I20260810 18:57:31.279207 25605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:31.287048 25605 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:31.287968 25605 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:31.306185 25605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620: Generated new TSK 0
I20260810 18:57:31.307046 25605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:31.315936 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39197
--builtin_ntp_servers=127.30.153.148:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:57:31.433285 25622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:31.433533 25622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:31.433571 25622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:31.437711 25622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:31.437760 25622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:31.437813 25622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:57:31.442478 25622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:31.443554 25622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:31.444942 25622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:31.454736 25629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:31.454736 25631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:31.454753 25633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:31.454840 25622 server_base.cc:1034] running on GCE node
I20260810 18:57:31.455432 25622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:31.456151 25622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:31.457378 25622 hybrid_clock.cc:648] HybridClock initialized: now 1786388251457362 us; error 26 us; skew 500 ppm
I20260810 18:57:31.459524 25622 webserver.cc:460] Webserver started at http://127.30.153.129:45389/ using document root <none> and password file <none>
I20260810 18:57:31.460131 25622 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:31.460232 25622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:31.460472 25622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:31.462604 25622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "d714e1422f3c4a609fd4f85e54dea2d8"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "70ae185dac75b5cc0e64676afed1b821"
server_key_iv: "d0819697e86e733777ef5a278c50bfb6"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.463225 25622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "d714e1422f3c4a609fd4f85e54dea2d8"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "70ae185dac75b5cc0e64676afed1b821"
server_key_iv: "d0819697e86e733777ef5a278c50bfb6"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.466980 25622 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:57:31.469726 25644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.471227 25622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:31.471385 25622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "d714e1422f3c4a609fd4f85e54dea2d8"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "70ae185dac75b5cc0e64676afed1b821"
server_key_iv: "d0819697e86e733777ef5a278c50bfb6"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.471516 25622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:31.509778 25622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:31.510713 25622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:31.510944 25622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:31.511623 25622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:31.512699 25622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:31.512779 25622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.512859 25622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:31.512918 25622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.523371 25622 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:33485
I20260810 18:57:31.523404 25815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:33485 every 8 connection(s)
I20260810 18:57:31.524370 25622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:57:31.534322 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.526364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:40775 (local address 127.30.153.190:39197)
0810 18:57:31.526521 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:31.526525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:31.527786 (+ 1261us) server_negotiation.cc:407] Connection header received
0810 18:57:31.529348 (+ 1562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:31.529350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:31.529399 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:31.529489 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:31.530541 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.531075 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.532241 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.532451 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.532535 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.533798 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:31.533801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:31.533803 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:31.533815 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:31.533835 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:31.533843 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:31.533906 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:31.534059 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 18:57:31.534142 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:57:31.534401 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 25622
I20260810 18:57:31.534541 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:57:31.534843 31334 external_mini_cluster.cc:1428] Setting key 5a843277865f9fe6244e4d40d4fb920b
I20260810 18:57:31.535341 25824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.526810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:39197 (local address 127.30.153.129:40775)
0810 18:57:31.527616 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:31.527673 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:31.529135 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:31.529699 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:31.529705 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:31.529788 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:31.530324 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.530339 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.531239 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.531242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.532111 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.532121 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.532615 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.532619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.532785 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.533519 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:31.533520 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:31.533562 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:31.533633 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:31.533927 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:31.533934 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:31.533939 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 18:57:31.534040 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 18:57:31.534303 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":161,"threads_started":1}
I20260810 18:57:31.536830 25816 heartbeater.cc:347] Connected to a master server at 127.30.153.190:39197
I20260810 18:57:31.537120 25816 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:31.537146 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39197
--builtin_ntp_servers=127.30.153.148:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:57:31.537738 25816 heartbeater.cc:511] Master 127.30.153.190:39197 requested a full tablet report, sending...
I20260810 18:57:31.539417 25528 ts_manager.cc:194] Registered new tserver with Master: d714e1422f3c4a609fd4f85e54dea2d8 (127.30.153.129:33485)
I20260810 18:57:31.540889 25528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.129:40775
W20260810 18:57:31.658052 25825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:31.658303 25825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:31.658339 25825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:31.662336 25825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:31.662392 25825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:31.662510 25825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:57:31.667768 25825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:31.668867 25825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:31.670303 25825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:31.680228 25831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:31.680238 25834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:31.680321 25825 server_base.cc:1034] running on GCE node
W20260810 18:57:31.680251 25837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:31.680884 25825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:31.681542 25825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:31.682706 25825 hybrid_clock.cc:648] HybridClock initialized: now 1786388251682692 us; error 28 us; skew 500 ppm
I20260810 18:57:31.684590 25825 webserver.cc:460] Webserver started at http://127.30.153.130:42405/ using document root <none> and password file <none>
I20260810 18:57:31.685228 25825 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:31.685298 25825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:31.685520 25825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:31.687660 25825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "6ee7ed8e19154c5683f9b98193dbe152"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "8a1eae4eede7767e9ed2307daf4681c0"
server_key_iv: "9e9e552c98b32622a000237fdb2e13ba"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.688227 25825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "6ee7ed8e19154c5683f9b98193dbe152"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "8a1eae4eede7767e9ed2307daf4681c0"
server_key_iv: "9e9e552c98b32622a000237fdb2e13ba"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.691898 25825 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 18:57:31.694316 25846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.695600 25825 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:57:31.695744 25825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "6ee7ed8e19154c5683f9b98193dbe152"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "8a1eae4eede7767e9ed2307daf4681c0"
server_key_iv: "9e9e552c98b32622a000237fdb2e13ba"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.695852 25825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:31.722999 25825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:31.723747 25825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:31.723959 25825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:31.724591 25825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:31.725584 25825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:31.725632 25825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.725710 25825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:31.725755 25825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.734869 25825 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:46601
I20260810 18:57:31.734895 26028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:46601 every 8 connection(s)
I20260810 18:57:31.735810 25825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:57:31.744967 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.737721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:47843 (local address 127.30.153.190:39197)
0810 18:57:31.737866 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:31.737870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:31.738928 (+ 1058us) server_negotiation.cc:407] Connection header received
0810 18:57:31.740356 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:31.740359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:31.740409 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:31.740494 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:31.741369 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.741919 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.743119 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.743374 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.743468 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.744540 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:31.744543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:31.744545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:31.744557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:31.744574 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:31.744582 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:31.744643 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:31.744793 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 18:57:31.744848 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:57:31.745512 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 25825
I20260810 18:57:31.745666 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:57:31.745913 26031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.738081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:39197 (local address 127.30.153.130:47843)
0810 18:57:31.738793 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:31.738836 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:31.740123 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:31.740657 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:31.740664 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:31.740760 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:31.741232 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.741243 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.742035 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.742037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.742991 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.743001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.743517 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.743520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.743655 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.744299 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:31.744300 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:31.744340 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:31.744403 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:31.744660 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:31.744667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:31.744671 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:57:31.744775 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 18:57:31.744983 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
I20260810 18:57:31.745980 31334 external_mini_cluster.cc:1428] Setting key a0348464c7cd5c54b4f81a57856cabea
I20260810 18:57:31.747123 26029 heartbeater.cc:347] Connected to a master server at 127.30.153.190:39197
I20260810 18:57:31.747368 26029 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:31.747996 26029 heartbeater.cc:511] Master 127.30.153.190:39197 requested a full tablet report, sending...
I20260810 18:57:31.748410 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39197
--builtin_ntp_servers=127.30.153.148:39297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:57:31.749051 25528 ts_manager.cc:194] Registered new tserver with Master: 6ee7ed8e19154c5683f9b98193dbe152 (127.30.153.130:46601)
I20260810 18:57:31.749614 25528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.130:47843
W20260810 18:57:31.864576 26033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:31.864835 26033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:31.864876 26033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:31.868904 26033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:31.868996 26033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:31.869099 26033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:57:31.874040 26033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:39197
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:31.875234 26033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:31.876758 26033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:31.886693 26055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:31.886729 26048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:31.886740 26047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:31.887245 26033 server_base.cc:1034] running on GCE node
I20260810 18:57:31.887712 26033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:31.888427 26033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:31.889653 26033 hybrid_clock.cc:648] HybridClock initialized: now 1786388251889632 us; error 36 us; skew 500 ppm
I20260810 18:57:31.891736 26033 webserver.cc:460] Webserver started at http://127.30.153.131:38077/ using document root <none> and password file <none>
I20260810 18:57:31.892376 26033 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:31.892479 26033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:31.892747 26033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:31.894980 26033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "b397f5841dca4d9f9c717e391555f400"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "1d7163fe8e3222d97acf3769b07e6e7e"
server_key_iv: "59f7eb494a5b4ffc468368ef588c1bfc"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.895602 26033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "b397f5841dca4d9f9c717e391555f400"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "1d7163fe8e3222d97acf3769b07e6e7e"
server_key_iv: "59f7eb494a5b4ffc468368ef588c1bfc"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.899605 26033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:57:31.902230 26062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.903674 26033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:31.903863 26033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "b397f5841dca4d9f9c717e391555f400"
format_stamp: "Formatted at 2026-08-10 18:57:31 on dist-test-slave-68xk"
server_key: "1d7163fe8e3222d97acf3769b07e6e7e"
server_key_iv: "59f7eb494a5b4ffc468368ef588c1bfc"
server_key_version: "encryptionkey@0"
I20260810 18:57:31.903996 26033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:31.919953 26033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:31.920694 26033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:31.920912 26033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:31.921550 26033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:31.922607 26033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:31.922683 26033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.922752 26033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:31.922806 26033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:31.932909 26033 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:35387
I20260810 18:57:31.932943 26256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:35387 every 8 connection(s)
I20260810 18:57:31.933931 26033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:57:31.936470 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 26033
I20260810 18:57:31.936614 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:57:31.936898 31334 external_mini_cluster.cc:1428] Setting key 375b49d4a41808f350e51d439a544454
I20260810 18:57:31.944065 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.936134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:59683 (local address 127.30.153.190:39197)
0810 18:57:31.936336 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:31.936344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:31.937523 (+ 1179us) server_negotiation.cc:407] Connection header received
0810 18:57:31.939273 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:31.939276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:31.939332 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:31.939413 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:31.940338 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.940875 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.942043 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.942284 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.942402 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.943555 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:31.943559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:31.943561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:31.943573 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:31.943592 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:31.943600 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:31.943685 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:31.943854 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 18:57:31.943918 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:57:31.944940 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.936568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:39197 (local address 127.30.153.131:59683)
0810 18:57:31.937359 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:31.937413 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:31.939036 (+ 1623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:31.939602 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:31.939609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:31.939702 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:31.940193 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.940206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.941013 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.941016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.941913 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.941922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.942469 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.942474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.942656 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.943283 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:31.943284 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:31.943324 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:31.943388 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:31.943699 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:31.943711 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:31.943717 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 18:57:31.943853 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:57:31.944138 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":140,"threads_started":1}
I20260810 18:57:31.946182 26258 heartbeater.cc:347] Connected to a master server at 127.30.153.190:39197
I20260810 18:57:31.946483 26258 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:31.947036 26258 heartbeater.cc:511] Master 127.30.153.190:39197 requested a full tablet report, sending...
I20260810 18:57:31.948140 25528 ts_manager.cc:194] Registered new tserver with Master: b397f5841dca4d9f9c717e391555f400 (127.30.153.131:35387)
I20260810 18:57:31.948770 25528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.131:59683
I20260810 18:57:31.951385 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:57:31.957296 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.953453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56978 (local address 127.30.153.190:39197)
0810 18:57:31.953572 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:31.953577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:31.953829 (+ 252us) server_negotiation.cc:407] Connection header received
0810 18:57:31.953903 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:31.953906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:31.953947 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:31.954021 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:31.954374 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.954956 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.955838 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.956041 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.956119 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.956921 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:31.956924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:31.956926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:31.956937 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:31.956955 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:31.956962 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:31.957020 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:31.957119 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 18:57:31.957167 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:57:31.960347 25528 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:57:31.963526 25528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:57:31.978633 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.972389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.30.153.130:46601)
0810 18:57:31.973798 (+ 1409us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:31.973811 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:31.973837 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:57:31.973921 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:31.973926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:31.974147 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:31.974310 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:31.974956 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.975871 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.977282 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.977878 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.978003 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.978184 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:31.978337 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 18:57:31.978463 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1282,"thread_start_us":113,"threads_started":1}
I20260810 18:57:31.978634 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.972244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:46601 (local address 127.0.0.1:49436)
0810 18:57:31.972894 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:31.972945 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:31.973085 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:31.974511 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:31.974518 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:31.974584 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:31.974795 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.974804 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.975991 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.975994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.977138 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.977149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.977998 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.978001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.978122 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.978166 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 18:57:31.978291 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 18:57:31.978430 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":113,"spinlock_wait_cycles":53888,"thread_start_us":97,"threads_started":1}
I20260810 18:57:31.979804 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.973073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.30.153.131:35387)
0810 18:57:31.973507 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:31.973520 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:31.974074 (+ 554us) server_negotiation.cc:407] Connection header received
0810 18:57:31.974310 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:31.974315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:31.974511 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:31.974636 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:31.975070 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.976219 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.977860 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.978733 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.978873 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.979051 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:31.979540 (+ 489us) server_negotiation.cc:299] Negotiation successful
0810 18:57:31.979657 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":112,"threads_started":1}
I20260810 18:57:31.980152 26275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.972550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35387 (local address 127.0.0.1:47744)
0810 18:57:31.973984 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:31.973996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:31.974142 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:31.974832 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:31.974837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:31.974860 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:31.974947 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.974952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.976376 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.976379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.977714 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.977725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.979267 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.979272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.979363 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.979386 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:57:31.979970 (+ 584us) client_negotiation.cc:240] Negotiation successful
0810 18:57:31.980052 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1369,"mutex_wait_us":70,"thread_start_us":98,"threads_started":1}
I20260810 18:57:31.982102 25925 tablet_service.cc:1467] Processing CreateTablet for tablet e5b78901b0c041ef8857983da2fdcfe2 (DEFAULT_TABLE table=test-table [id=3c29e554b32d49c0a87686fde9f80200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:31.983032 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.973074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57696 (local address 127.30.153.129:33485)
0810 18:57:31.973566 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:31.973572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:31.973841 (+ 269us) server_negotiation.cc:407] Connection header received
0810 18:57:31.973953 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:31.973961 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:31.974183 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:31.974349 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:31.974878 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.976205 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.977872 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.979366 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.979486 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.979641 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:31.982738 (+ 3097us) server_negotiation.cc:299] Negotiation successful
0810 18:57:31.982847 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":126,"threads_started":1}
I20260810 18:57:31.983362 25925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b78901b0c041ef8857983da2fdcfe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:31.983016 26134 tablet_service.cc:1467] Processing CreateTablet for tablet e5b78901b0c041ef8857983da2fdcfe2 (DEFAULT_TABLE table=test-table [id=3c29e554b32d49c0a87686fde9f80200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:31.983685 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:31.972298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:33485 (local address 127.0.0.1:57696)
0810 18:57:31.973672 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:31.973686 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:31.973798 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:31.974516 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:31.974522 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:31.974559 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:31.974748 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.974754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.976354 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.976357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.977714 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:31.977725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:31.982493 (+ 4768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:31.982501 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:31.982575 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:31.982595 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:57:31.982697 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:57:31.983577 (+ 880us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"mutex_wait_us":117,"spinlock_wait_cycles":768,"thread_start_us":119,"threads_started":1}
I20260810 18:57:31.984087 26134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b78901b0c041ef8857983da2fdcfe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:31.986037 25719 tablet_service.cc:1467] Processing CreateTablet for tablet e5b78901b0c041ef8857983da2fdcfe2 (DEFAULT_TABLE table=test-table [id=3c29e554b32d49c0a87686fde9f80200]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:31.987031 25719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5b78901b0c041ef8857983da2fdcfe2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:31.991398 26282 tablet_bootstrap.cc:492] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: Bootstrap starting.
I20260810 18:57:31.991845 26283 tablet_bootstrap.cc:492] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: Bootstrap starting.
I20260810 18:57:31.993404 26284 tablet_bootstrap.cc:492] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8: Bootstrap starting.
I20260810 18:57:31.995440 26282 tablet_bootstrap.cc:654] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:31.995450 26283 tablet_bootstrap.cc:654] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:31.995574 26284 tablet_bootstrap.cc:654] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:31.996290 26282 log.cc:824] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:31.996366 26284 log.cc:824] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:31.996366 26283 log.cc:824] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:31.998783 26284 tablet_bootstrap.cc:492] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8: No bootstrap required, opened a new log
I20260810 18:57:31.999066 26284 ts_tablet_manager.cc:1397] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260810 18:57:31.999285 26282 tablet_bootstrap.cc:492] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: No bootstrap required, opened a new log
I20260810 18:57:31.999285 26283 tablet_bootstrap.cc:492] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: No bootstrap required, opened a new log
I20260810 18:57:31.999473 26282 ts_tablet_manager.cc:1397] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 18:57:31.999473 26283 ts_tablet_manager.cc:1397] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260810 18:57:32.002934 26284 raft_consensus.cc:348] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.003157 26284 raft_consensus.cc:374] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:32.003188 26284 raft_consensus.cc:729] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d714e1422f3c4a609fd4f85e54dea2d8, State: Initialized, Role: FOLLOWER
I20260810 18:57:32.003613 26284 consensus_queue.cc:260] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.003858 26282 raft_consensus.cc:348] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.003858 26283 raft_consensus.cc:348] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.004106 26283 raft_consensus.cc:374] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:32.004106 26282 raft_consensus.cc:374] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:32.004158 26283 raft_consensus.cc:729] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b397f5841dca4d9f9c717e391555f400, State: Initialized, Role: FOLLOWER
I20260810 18:57:32.004158 26282 raft_consensus.cc:729] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee7ed8e19154c5683f9b98193dbe152, State: Initialized, Role: FOLLOWER
I20260810 18:57:32.004302 25816 heartbeater.cc:503] Master 127.30.153.190:39197 was elected leader, sending a full tablet report...
I20260810 18:57:32.004595 26283 consensus_queue.cc:260] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.004688 26284 ts_tablet_manager.cc:1428] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260810 18:57:32.005182 26282 consensus_queue.cc:260] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.005292 26258 heartbeater.cc:503] Master 127.30.153.190:39197 was elected leader, sending a full tablet report...
I20260810 18:57:32.005887 26283 ts_tablet_manager.cc:1428] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:57:32.006306 26282 ts_tablet_manager.cc:1428] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:57:32.006601 26029 heartbeater.cc:503] Master 127.30.153.190:39197 was elected leader, sending a full tablet report...
W20260810 18:57:32.025666 25822 tablet.cc:2369] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:57:32.091241 26290 raft_consensus.cc:482] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:32.091572 26290 raft_consensus.cc:504] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.093147 26290 leader_election.cc:288] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ee7ed8e19154c5683f9b98193dbe152 (127.30.153.130:46601), b397f5841dca4d9f9c717e391555f400 (127.30.153.131:35387)
I20260810 18:57:32.097337 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:32.093461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35387 (local address 127.30.153.129:48995)
0810 18:57:32.093920 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:32.093933 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:32.094068 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:32.094387 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:32.094390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:32.094408 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:32.094573 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:32.094579 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.095340 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.095345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:32.096180 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:32.096189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.096867 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.096871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:32.096980 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:32.097011 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 18:57:32.097106 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 18:57:32.097176 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":130,"threads_started":1}
I20260810 18:57:32.097321 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:32.093561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:48995 (local address 127.30.153.131:35387)
0810 18:57:32.093719 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:32.093725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:32.094020 (+ 295us) server_negotiation.cc:407] Connection header received
0810 18:57:32.094193 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:32.094195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:32.094248 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:32.094362 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:32.094690 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.095208 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.096321 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.096728 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.096869 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:32.096921 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:32.097124 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 18:57:32.097197 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 18:57:32.097570 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:32.093695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:36751 (local address 127.30.153.130:46601)
0810 18:57:32.093822 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:32.093826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:32.094195 (+ 369us) server_negotiation.cc:407] Connection header received
0810 18:57:32.094336 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:32.094340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:32.094398 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:32.094520 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:32.094856 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.095403 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.096533 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.096969 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.097110 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:32.097161 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:32.097375 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 18:57:32.097450 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:57:32.097594 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:32.093460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:46601 (local address 127.30.153.129:36751)
0810 18:57:32.094070 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:32.094084 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:32.094226 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:32.094552 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:32.094555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:32.094574 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:32.094707 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:32.094720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.095534 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.095537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:32.096391 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:32.096400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.097106 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.097110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:32.097237 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:32.097258 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:57:32.097356 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 18:57:32.097431 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":149,"thread_start_us":137,"threads_started":1}
I20260810 18:57:32.097980 26174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b78901b0c041ef8857983da2fdcfe2" candidate_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b397f5841dca4d9f9c717e391555f400" is_pre_election: true
I20260810 18:57:32.098304 26174 raft_consensus.cc:2454] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d714e1422f3c4a609fd4f85e54dea2d8 in term 0.
I20260810 18:57:32.098335 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b78901b0c041ef8857983da2fdcfe2" candidate_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ee7ed8e19154c5683f9b98193dbe152" is_pre_election: true
I20260810 18:57:32.098707 25962 raft_consensus.cc:2454] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d714e1422f3c4a609fd4f85e54dea2d8 in term 0.
I20260810 18:57:32.099079 25645 leader_election.cc:302] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b397f5841dca4d9f9c717e391555f400, d714e1422f3c4a609fd4f85e54dea2d8; no voters:
I20260810 18:57:32.099382 26290 raft_consensus.cc:2781] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:57:32.099478 26290 raft_consensus.cc:482] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:57:32.099522 26290 raft_consensus.cc:3037] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:32.100682 26290 raft_consensus.cc:504] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.101033 26290 leader_election.cc:288] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [CANDIDATE]: Term 1 election: Requested vote from peers 6ee7ed8e19154c5683f9b98193dbe152 (127.30.153.130:46601), b397f5841dca4d9f9c717e391555f400 (127.30.153.131:35387)
I20260810 18:57:32.101397 26174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b78901b0c041ef8857983da2fdcfe2" candidate_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b397f5841dca4d9f9c717e391555f400"
I20260810 18:57:32.101436 25962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5b78901b0c041ef8857983da2fdcfe2" candidate_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ee7ed8e19154c5683f9b98193dbe152"
I20260810 18:57:32.101511 26174 raft_consensus.cc:3037] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:32.101610 25962 raft_consensus.cc:3037] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:32.102567 26174 raft_consensus.cc:2454] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d714e1422f3c4a609fd4f85e54dea2d8 in term 1.
I20260810 18:57:32.102891 25645 leader_election.cc:302] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b397f5841dca4d9f9c717e391555f400, d714e1422f3c4a609fd4f85e54dea2d8; no voters:
I20260810 18:57:32.102955 25962 raft_consensus.cc:2454] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d714e1422f3c4a609fd4f85e54dea2d8 in term 1.
I20260810 18:57:32.103070 26290 raft_consensus.cc:2781] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:32.103327 26290 raft_consensus.cc:686] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 1 LEADER]: Becoming Leader. State: Replica: d714e1422f3c4a609fd4f85e54dea2d8, State: Running, Role: LEADER
I20260810 18:57:32.103662 26290 consensus_queue.cc:237] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } }
I20260810 18:57:32.107257 25529 catalog_manager.cc:5495] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d714e1422f3c4a609fd4f85e54dea2d8 (127.30.153.129). New cstate: current_term: 1 leader_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d714e1422f3c4a609fd4f85e54dea2d8" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 33485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 } health_report { overall_health: UNKNOWN } } }
I20260810 18:57:32.131457 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:32.127553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57710 (local address 127.30.153.129:33485)
0810 18:57:32.127676 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:32.127681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:32.127721 (+ 40us) server_negotiation.cc:407] Connection header received
0810 18:57:32.127951 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:32.127953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:32.128013 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:32.128130 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:57:32.128404 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.129019 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.129899 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:32.130125 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:32.130237 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:32.130612 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:57:32.131021 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:57:32.131168 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:32.131249 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 18:57:32.131306 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:57:32.137614 26174 raft_consensus.cc:1261] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 1 FOLLOWER]: Refusing update from remote peer d714e1422f3c4a609fd4f85e54dea2d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:32.137614 25962 raft_consensus.cc:1261] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 1 FOLLOWER]: Refusing update from remote peer d714e1422f3c4a609fd4f85e54dea2d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:32.138373 26290 consensus_queue.cc:1035] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b397f5841dca4d9f9c717e391555f400" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:32.138604 26302 consensus_queue.cc:1035] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ee7ed8e19154c5683f9b98193dbe152" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 46601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:32.149684 26312 mvcc.cc:205] Tried to move back new op lower bound from 7317046280747708416 to 7317046280619368448. Current Snapshot: MvccSnapshot[applied={T|T < 7317046280747708416}]
I20260810 18:57:32.156414 25528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:32.156575 25528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:32.159231 25528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00af0e9de39246419d585ef6f64f0620: Sending DeleteTablet for 3 replicas of tablet e5b78901b0c041ef8857983da2fdcfe2
I20260810 18:57:32.160104 25719 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b78901b0c041ef8857983da2fdcfe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:32 UTC) from {username='slave'} at 127.0.0.1:57696
I20260810 18:57:32.160106 25925 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b78901b0c041ef8857983da2fdcfe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:32 UTC) from {username='slave'} at 127.0.0.1:49436
I20260810 18:57:32.160260 26134 tablet_service.cc:1514] Processing DeleteTablet for tablet e5b78901b0c041ef8857983da2fdcfe2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:32 UTC) from {username='slave'} at 127.0.0.1:47744
I20260810 18:57:32.160781 26326 tablet_replica.cc:330] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8: stopping tablet replica
I20260810 18:57:32.161099 26326 raft_consensus.cc:2229] T e5b78901b0c041ef8857983da2fdcfe2 P d714e1422f3c4a609fd4f85e54dea2d8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:57:32.161377 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 25622
I20260810 18:57:32.164573 26327 tablet_replica.cc:330] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: stopping tablet replica
I20260810 18:57:32.164951 26327 raft_consensus.cc:2229] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:32.165349 26327 raft_consensus.cc:2258] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:32.161237 26328 tablet_replica.cc:330] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: stopping tablet replica
I20260810 18:57:32.169183 26328 raft_consensus.cc:2229] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:32.169495 26328 raft_consensus.cc:2258] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:32.170935 26327 ts_tablet_manager.cc:1905] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:32.171264 26328 ts_tablet_manager.cc:1905] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:32.174108 26327 ts_tablet_manager.cc:1918] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:32.174108 26328 ts_tablet_manager.cc:1918] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:32.174232 26327 log.cc:1196] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/e5b78901b0c041ef8857983da2fdcfe2
I20260810 18:57:32.174239 26328 log.cc:1196] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/e5b78901b0c041ef8857983da2fdcfe2
I20260810 18:57:32.174642 26328 ts_tablet_manager.cc:1939] T e5b78901b0c041ef8857983da2fdcfe2 P b397f5841dca4d9f9c717e391555f400: Deleting consensus metadata
I20260810 18:57:32.174661 26327 ts_tablet_manager.cc:1939] T e5b78901b0c041ef8857983da2fdcfe2 P 6ee7ed8e19154c5683f9b98193dbe152: Deleting consensus metadata
W20260810 18:57:32.175076 25509 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:33485 (error 108)
I20260810 18:57:32.176074 25507 catalog_manager.cc:4841] TS b397f5841dca4d9f9c717e391555f400 (127.30.153.131:35387): tablet e5b78901b0c041ef8857983da2fdcfe2 (table test-table [id=3c29e554b32d49c0a87686fde9f80200]) successfully deleted
I20260810 18:57:32.176119 25511 catalog_manager.cc:4841] TS 6ee7ed8e19154c5683f9b98193dbe152 (127.30.153.130:46601): tablet e5b78901b0c041ef8857983da2fdcfe2 (table test-table [id=3c29e554b32d49c0a87686fde9f80200]) successfully deleted
I20260810 18:57:32.176270 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:32.175895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:33485 (local address 127.0.0.1:57724)
0810 18:57:32.176060 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:32.176163 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:33485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 18:57:32.176405 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 25825
W20260810 18:57:32.176489 25509 catalog_manager.cc:4584] TS d714e1422f3c4a609fd4f85e54dea2d8 (127.30.153.129:33485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5b78901b0c041ef8857983da2fdcfe2: Network error: Client connection negotiation failed: client connection to 127.30.153.129:33485: connect: Connection refused (error 111)
I20260810 18:57:32.191746 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 26033
I20260810 18:57:32.205821 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 25477
2026-08-10T18:57:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1164 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:32 dist-test-slave-68xk krb5kdc[26335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:32 dist-test-slave-68xk krb5kdc[26335](info): set up 2 sockets
Aug 10 18:57:32 dist-test-slave-68xk krb5kdc[26335](info): commencing operation
krb5kdc: starting...
W20260810 18:57:33.483790 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:33 dist-test-slave-68xk krb5kdc[26335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:57:33.639300 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:33925
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:33925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:33.754153 26351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:33.754492 26351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:33.754609 26351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:33.758565 26351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:33.758657 26351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:33.758697 26351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:33.758733 26351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:33.758776 26351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:33.764068 26351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:33925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:33925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:33.765180 26351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:33.766252 26351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:33.775039 26364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:33.775039 26359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:33.775069 26351 server_base.cc:1034] running on GCE node
W20260810 18:57:33.775117 26360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:33.776011 26351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:33.777035 26351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:33.778188 26351 hybrid_clock.cc:648] HybridClock initialized: now 1786388253778157 us; error 47 us; skew 500 ppm
Aug 10 18:57:33 dist-test-slave-68xk krb5kdc[26335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388253, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:33.781077 26351 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:57:33.782003 26351 webserver.cc:460] Webserver started at http://127.30.153.190:38395/ using document root <none> and password file <none>
I20260810 18:57:33.782635 26351 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:33.782734 26351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:33.782975 26351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:33.785044 26351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "05b876a33eca4272815d66b6730f4720"
format_stamp: "Formatted at 2026-08-10 18:57:33 on dist-test-slave-68xk"
server_key: "a9b00a8b9d65b784e6621a5d98b6a537"
server_key_iv: "6d531840628936402897cecc3ae1b58e"
server_key_version: "encryptionkey@0"
I20260810 18:57:33.785688 26351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "05b876a33eca4272815d66b6730f4720"
format_stamp: "Formatted at 2026-08-10 18:57:33 on dist-test-slave-68xk"
server_key: "a9b00a8b9d65b784e6621a5d98b6a537"
server_key_iv: "6d531840628936402897cecc3ae1b58e"
server_key_version: "encryptionkey@0"
I20260810 18:57:33.789505 26351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:57:33.792062 26377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:33.793298 26351 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:57:33.793447 26351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "05b876a33eca4272815d66b6730f4720"
format_stamp: "Formatted at 2026-08-10 18:57:33 on dist-test-slave-68xk"
server_key: "a9b00a8b9d65b784e6621a5d98b6a537"
server_key_iv: "6d531840628936402897cecc3ae1b58e"
server_key_version: "encryptionkey@0"
I20260810 18:57:33.793560 26351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:33.808360 26351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:33.811651 26351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:33.811883 26351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:33.819361 26351 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:33925
I20260810 18:57:33.819391 26472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:33925 every 8 connection(s)
I20260810 18:57:33.820335 26351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:33.823127 26473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:33.826043 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 26351
I20260810 18:57:33.826192 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:33.826537 31334 external_mini_cluster.cc:1428] Setting key 839a20a1b74f9daecc483077b29c8f1d
I20260810 18:57:33.829365 26473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720: Bootstrap starting.
I20260810 18:57:33.831724 26473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:33.832496 26473 log.cc:824] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:33.834647 26473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720: No bootstrap required, opened a new log
Aug 10 18:57:33 dist-test-slave-68xk krb5kdc[26335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:33.838157 26473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b876a33eca4272815d66b6730f4720" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33925 } }
I20260810 18:57:33.838373 26473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:33.838476 26473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b876a33eca4272815d66b6730f4720, State: Initialized, Role: FOLLOWER
I20260810 18:57:33.838937 26473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b876a33eca4272815d66b6730f4720" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33925 } }
I20260810 18:57:33.839085 26473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:33.839179 26473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:33.839296 26473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:33.840426 26473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b876a33eca4272815d66b6730f4720" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33925 } }
I20260810 18:57:33.840808 26473 leader_election.cc:302] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05b876a33eca4272815d66b6730f4720; no voters:
I20260810 18:57:33.841213 26473 leader_election.cc:288] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:33.841464 26481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:33.841743 26481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [term 1 LEADER]: Becoming Leader. State: Replica: 05b876a33eca4272815d66b6730f4720, State: Running, Role: LEADER
I20260810 18:57:33.842089 26481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b876a33eca4272815d66b6730f4720" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33925 } }
I20260810 18:57:33.842391 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:33.828049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53084 (local address 127.30.153.190:33925)
0810 18:57:33.828519 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:33.828535 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:33.828570 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:57:33.829177 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:33.829194 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:33.829552 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:33.829903 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:33.830819 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:33.831930 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:33.832865 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:33.833343 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:33.833515 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:33.836379 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:33.836385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:33.836397 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:33.836422 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:33.838907 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:33.839461 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:33.839467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:33.839473 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:33.839627 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:33.839930 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:33.839933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:33.839935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:33.840505 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:33.840730 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:33.841028 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 18:57:33.841375 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":142,"threads_started":1}
I20260810 18:57:33.842499 26473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:33.843837 26482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05b876a33eca4272815d66b6730f4720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b876a33eca4272815d66b6730f4720" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33925 } } }
I20260810 18:57:33.843978 26482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:33.844331 26484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05b876a33eca4272815d66b6730f4720. Latest consensus state: current_term: 1 leader_uuid: "05b876a33eca4272815d66b6730f4720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05b876a33eca4272815d66b6730f4720" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 33925 } } }
I20260810 18:57:33.844424 26484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:33.844645 26492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:33.847692 26492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:33.853420 26492 catalog_manager.cc:1347] Generated new cluster ID: 7abb78a1fded4330ba77eaa59e93080c
I20260810 18:57:33.853487 26492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:33.867625 26492 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:33.868643 26492 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:33.879777 26492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720: Generated new TSK 0
I20260810 18:57:33.880538 26492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:57:33.918623 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33925
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:34.057806 26513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:34.058051 26513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:34.058094 26513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:34.062031 26513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:34.062086 26513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:34.062186 26513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:57:34.067723 26513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:34.068722 26513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:34.069742 26513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:34.079581 26524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:34.079627 26513 server_base.cc:1034] running on GCE node
W20260810 18:57:34.079600 26522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:34.079705 26521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:34.080193 26513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:34.080837 26513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:34.082026 26513 hybrid_clock.cc:648] HybridClock initialized: now 1786388254082012 us; error 26 us; skew 500 ppm
Aug 10 18:57:34 dist-test-slave-68xk krb5kdc[26335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388254, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:34.084805 26513 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:57:34.085685 26513 webserver.cc:460] Webserver started at http://127.30.153.129:35233/ using document root <none> and password file <none>
I20260810 18:57:34.086288 26513 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:34.086387 26513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:34.086675 26513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:34.088728 26513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "18729cbdb3db471a81c71f0d22417501"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "3e82ea558192a1e766c23c3b6ea6469b"
server_key_iv: "d3e394b963f3e6df9d5a00b088e2831f"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.089314 26513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "18729cbdb3db471a81c71f0d22417501"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "3e82ea558192a1e766c23c3b6ea6469b"
server_key_iv: "d3e394b963f3e6df9d5a00b088e2831f"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.093056 26513 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 18:57:34.095557 26533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.096808 26513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:57:34.096942 26513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "18729cbdb3db471a81c71f0d22417501"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "3e82ea558192a1e766c23c3b6ea6469b"
server_key_iv: "d3e394b963f3e6df9d5a00b088e2831f"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.097057 26513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:34.127238 26513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:34.130221 26513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:34.130471 26513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:34.131111 26513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:34.131997 26513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:34.132071 26513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.132146 26513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:34.132191 26513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.141078 26513 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:36117
I20260810 18:57:34.141108 26712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:36117 every 8 connection(s)
I20260810 18:57:34.141968 26513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:57:34.147359 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 26513
I20260810 18:57:34.147514 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:57:34.147897 31334 external_mini_cluster.cc:1428] Setting key 14a8c07fabb88bcd4ce81611448c6cb1
Aug 10 18:57:34 dist-test-slave-68xk krb5kdc[26335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388254, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:34.157915 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.143911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:46845 (local address 127.30.153.190:33925)
0810 18:57:34.144048 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.144054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.144906 (+ 852us) server_negotiation.cc:407] Connection header received
0810 18:57:34.146294 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.146296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.146346 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.146478 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:34.147787 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.148530 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.150260 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.150685 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.150822 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.153782 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:34.153789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:34.153795 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:34.153830 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:34.155721 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:34.156225 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:34.156232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:34.156236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:34.156355 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:34.156700 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:34.156706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:34.156710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:34.156981 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:34.157091 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.157590 (+ 499us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.157727 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 18:57:34.158720 26718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.144277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33925 (local address 127.30.153.129:46845)
0810 18:57:34.144746 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.144791 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:34.146090 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:34.146626 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:34.146633 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:34.147000 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:34.147567 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.147604 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.148738 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.148742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.150101 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.150115 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.150863 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.150867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.151024 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.151767 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:34.151793 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:34.153599 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:34.155841 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:34.155845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:34.155857 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:34.156112 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:34.156464 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:34.156467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:34.156468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:34.156600 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:34.157095 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:34.157100 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:34.157335 (+ 235us) client_negotiation.cc:769] Sending connection context
0810 18:57:34.157490 (+ 155us) client_negotiation.cc:240] Negotiation successful
0810 18:57:34.157740 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":116,"threads_started":1}
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:57:34.160066 26716 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33925
I20260810 18:57:34.160317 26716 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:34.160861 26716 heartbeater.cc:511] Master 127.30.153.190:33925 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
I20260810 18:57:34.162577 26409 ts_manager.cc:194] Registered new tserver with Master: 18729cbdb3db471a81c71f0d22417501 (127.30.153.129:36117)
I20260810 18:57:34.163731 26409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:46845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:57:34.177606 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33925
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:34.291559 26722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:34.291851 26722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:34.291896 26722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:34.296073 26722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:34.296141 26722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:34.296234 26722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:57:34.302364 26722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:34.303658 26722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:34.304759 26722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:34.314896 26729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:34.314929 26733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:34.314935 26736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:34.315083 26722 server_base.cc:1034] running on GCE node
I20260810 18:57:34.315678 26722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:34.316382 26722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:34.317572 26722 hybrid_clock.cc:648] HybridClock initialized: now 1786388254317556 us; error 27 us; skew 500 ppm
Aug 10 18:57:34 dist-test-slave-68xk krb5kdc[26335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388254, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:34.320631 26722 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:57:34.321527 26722 webserver.cc:460] Webserver started at http://127.30.153.130:37779/ using document root <none> and password file <none>
I20260810 18:57:34.322057 26722 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:34.322127 26722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:34.322307 26722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:34.324437 26722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "2630a78187ad4accb4888d1b6534329d"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "5241477b2b9976827ee14efc4c652c7c"
server_key_iv: "efe3959ca6e1b7b1b61c9d8a53028767"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.324949 26722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "2630a78187ad4accb4888d1b6534329d"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "5241477b2b9976827ee14efc4c652c7c"
server_key_iv: "efe3959ca6e1b7b1b61c9d8a53028767"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.328872 26722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:57:34.331421 26747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.332746 26722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:57:34.332909 26722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "2630a78187ad4accb4888d1b6534329d"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "5241477b2b9976827ee14efc4c652c7c"
server_key_iv: "efe3959ca6e1b7b1b61c9d8a53028767"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.333026 26722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:34.349917 26722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:34.352795 26722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:34.352999 26722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:34.353585 26722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:34.354478 26722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:34.354547 26722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.354619 26722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:34.354663 26722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.363740 26722 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:38835
I20260810 18:57:34.363775 26927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:38835 every 8 connection(s)
I20260810 18:57:34.364625 26722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
Aug 10 18:57:34 dist-test-slave-68xk krb5kdc[26335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388254, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:34.374615 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 26722
I20260810 18:57:34.374752 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:57:34.375120 31334 external_mini_cluster.cc:1428] Setting key 786b6d5101b35ca854cb64d6664f0656
I20260810 18:57:34.378844 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.366606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:49619 (local address 127.30.153.190:33925)
0810 18:57:34.366748 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.366752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.367603 (+ 851us) server_negotiation.cc:407] Connection header received
0810 18:57:34.368885 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.368888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.368939 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.369014 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:34.370181 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.370818 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.372061 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.372288 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.372371 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.375014 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:34.375027 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:34.375030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:34.375059 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:34.376604 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:34.377129 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:34.377134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:34.377136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:34.377233 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:34.377568 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:34.377574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:34.377578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:34.377849 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:34.377958 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.378440 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.378589 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:57:34.379776 26930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.366957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33925 (local address 127.30.153.130:49619)
0810 18:57:34.367432 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.367475 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:34.368676 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:34.369178 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:34.369184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:34.369563 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:34.370029 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.370041 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.370933 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.370936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.371947 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.371957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.372421 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.372424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.372555 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.373152 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:34.373168 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:34.374780 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:34.376715 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:34.376721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:34.376734 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:34.377015 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:34.377335 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:34.377337 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:34.377338 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:34.377470 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:34.377955 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:34.377959 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:34.378177 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 18:57:34.378330 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 18:57:34.378666 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":120,"threads_started":1}
I20260810 18:57:34.381217 26928 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33925
I20260810 18:57:34.381465 26928 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:34.381976 26928 heartbeater.cc:511] Master 127.30.153.190:33925 requested a full tablet report, sending...
I20260810 18:57:34.383112 26409 ts_manager.cc:194] Registered new tserver with Master: 2630a78187ad4accb4888d1b6534329d (127.30.153.130:38835)
I20260810 18:57:34.383997 26409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:49619
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:57:34.405498 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33925
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:34.521726 26935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:34.522027 26935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:34.522104 26935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:34.526137 26935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:34.526234 26935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:34.526324 26935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:57:34.531656 26935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:33925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:34.532738 26935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:34.533828 26935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:34.544297 26940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:34.544454 26935 server_base.cc:1034] running on GCE node
W20260810 18:57:34.544337 26944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:34.544621 26942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:34.545172 26935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:34.545826 26935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:34.546995 26935 hybrid_clock.cc:648] HybridClock initialized: now 1786388254546976 us; error 32 us; skew 500 ppm
Aug 10 18:57:34 dist-test-slave-68xk krb5kdc[26335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388254, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:34.550164 26935 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:57:34.551167 26935 webserver.cc:460] Webserver started at http://127.30.153.131:46167/ using document root <none> and password file <none>
I20260810 18:57:34.551796 26935 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:34.551899 26935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:34.552161 26935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:34.554363 26935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "e6ffc0586db445a59d17392d6b61cd1e"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "2664d272cd7e409a6c1fd31e92ef1cfa"
server_key_iv: "366f345d029be632e9f37688cfc71c31"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.555030 26935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "e6ffc0586db445a59d17392d6b61cd1e"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "2664d272cd7e409a6c1fd31e92ef1cfa"
server_key_iv: "366f345d029be632e9f37688cfc71c31"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.559015 26935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:57:34.561654 26953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.563010 26935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:57:34.563182 26935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "e6ffc0586db445a59d17392d6b61cd1e"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "2664d272cd7e409a6c1fd31e92ef1cfa"
server_key_iv: "366f345d029be632e9f37688cfc71c31"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.563303 26935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:34.580427 26935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:34.583709 26935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:34.583907 26935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:34.584497 26935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:34.585388 26935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:34.585462 26935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.585533 26935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:34.585585 26935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.594399 26935 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:37941
I20260810 18:57:34.594488 27131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:37941 every 8 connection(s)
I20260810 18:57:34.595515 26935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:57:34.602247 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 26935
I20260810 18:57:34.602416 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:57:34.602885 31334 external_mini_cluster.cc:1428] Setting key 0c4ef858e7546ab04635f934b8c536d0
Aug 10 18:57:34 dist-test-slave-68xk krb5kdc[26335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388254, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:34.609930 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.597648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:35469 (local address 127.30.153.190:33925)
0810 18:57:34.597816 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.597820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.598783 (+ 963us) server_negotiation.cc:407] Connection header received
0810 18:57:34.600184 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.600187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.600244 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.600398 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:34.601661 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.602231 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.603748 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.604038 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.604163 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.606886 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:34.606889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:34.606891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:34.606915 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:34.608151 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:34.608627 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:34.608630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:34.608631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:34.608703 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:34.609012 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:34.609014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:34.609015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:34.609183 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:34.609263 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.609714 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.609815 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:57:34.610936 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.598054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:33925 (local address 127.30.153.131:35469)
0810 18:57:34.598622 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.598667 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:34.599956 (+ 1289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:34.600585 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:34.600592 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:34.601029 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:34.601507 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.601520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.602369 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.602373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.603625 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.603634 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.604331 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.604334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.604476 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.605152 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:34.605169 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:34.606732 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:34.608265 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:34.608269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:34.608281 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:34.608531 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:34.608780 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:34.608782 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:34.608784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:34.608917 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:34.609269 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:34.609274 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:34.609501 (+ 227us) client_negotiation.cc:769] Sending connection context
0810 18:57:34.609660 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 18:57:34.609907 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":136,"threads_started":1}
I20260810 18:57:34.612172 27134 heartbeater.cc:347] Connected to a master server at 127.30.153.190:33925
I20260810 18:57:34.612433 27134 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:34.612946 27134 heartbeater.cc:511] Master 127.30.153.190:33925 requested a full tablet report, sending...
I20260810 18:57:34.613973 26409 ts_manager.cc:194] Registered new tserver with Master: e6ffc0586db445a59d17392d6b61cd1e (127.30.153.131:37941)
I20260810 18:57:34.614672 26409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:35469
I20260810 18:57:34.616958 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:57:34.626559 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.619089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53090 (local address 127.30.153.190:33925)
0810 18:57:34.619245 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.619248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.619345 (+ 97us) server_negotiation.cc:407] Connection header received
0810 18:57:34.619446 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.619448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.619484 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.619565 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:34.620299 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.620857 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.621654 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.621883 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.621972 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.623588 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:34.623592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:34.623594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:34.623619 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:34.624908 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:34.625397 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:34.625400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:34.625401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:34.625473 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:34.625755 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:34.625757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:34.625759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:34.625944 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:34.626054 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.626286 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.626363 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:57:34.629276 26409 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53090:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:57:34.632241 26409 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:57:34.647583 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.641119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58388 (local address 127.30.153.130:38835)
0810 18:57:34.642728 (+ 1609us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.642741 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.642759 (+ 18us) server_negotiation.cc:407] Connection header received
0810 18:57:34.642838 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.642843 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.643035 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.643236 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:34.643624 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.644549 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.646074 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.646865 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.647002 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.647146 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.647337 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.647435 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1452,"thread_start_us":137,"threads_started":1}
I20260810 18:57:34.647576 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.640953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38835 (local address 127.0.0.1:58388)
0810 18:57:34.641859 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.641913 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:34.642058 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:34.643397 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:34.643401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:34.643420 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:34.643519 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.643525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.644713 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.644716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.645938 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.645948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.646998 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.647002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.647121 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.647161 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:57:34.647301 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 18:57:34.647425 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"mutex_wait_us":90,"spinlock_wait_cycles":9344,"thread_start_us":95,"threads_started":1}
I20260810 18:57:34.647898 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.641174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:37941 (local address 127.0.0.1:38428)
0810 18:57:34.641896 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.641913 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:34.642056 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:34.643208 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:34.643210 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:34.643229 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:34.643345 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.643352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.644976 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.644979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.646402 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.646434 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.647504 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.647508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.647596 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.647618 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:57:34.647674 (+ 56us) client_negotiation.cc:240] Negotiation successful
0810 18:57:34.647756 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":162,"spinlock_wait_cycles":13312,"thread_start_us":125,"threads_started":1}
I20260810 18:57:34.648320 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.641286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:36117 (local address 127.0.0.1:43022)
0810 18:57:34.642292 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.642309 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:34.642394 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:34.643025 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:34.643030 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:34.643176 (+ 146us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:34.643345 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.643352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.644962 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.644967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.646294 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.646305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.647898 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.647903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.648010 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.648038 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 18:57:34.648112 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 18:57:34.648190 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":105,"thread_start_us":104,"threads_started":1}
I20260810 18:57:34.648649 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.642148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38428 (local address 127.30.153.131:37941)
0810 18:57:34.642711 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.642725 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.642752 (+ 27us) server_negotiation.cc:407] Connection header received
0810 18:57:34.642863 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.642868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.643035 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.643236 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:34.643675 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.644840 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.646554 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.647358 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.648220 (+ 862us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.648345 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.648397 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.648507 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":161,"threads_started":1}
I20260810 18:57:34.650816 27161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.641708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43022 (local address 127.30.153.129:36117)
0810 18:57:34.642359 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.642378 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.642403 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:57:34.642575 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.642580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.642861 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.643024 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:34.643461 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.644790 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.646446 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.647358 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.647499 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.647622 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.650549 (+ 2927us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.650666 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":95,"threads_started":1}
I20260810 18:57:34.650825 26817 tablet_service.cc:1467] Processing CreateTablet for tablet 0469b367e21e450b854b89e56f4c161c (DEFAULT_TABLE table=test-table [id=c37fea8d387a44fe953e3a3bceaa365e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:34.651374 27034 tablet_service.cc:1467] Processing CreateTablet for tablet 0469b367e21e450b854b89e56f4c161c (DEFAULT_TABLE table=test-table [id=c37fea8d387a44fe953e3a3bceaa365e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:34.651942 26817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0469b367e21e450b854b89e56f4c161c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:34.652313 27034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0469b367e21e450b854b89e56f4c161c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:34.658936 27168 tablet_bootstrap.cc:492] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: Bootstrap starting.
I20260810 18:57:34.659222 27167 tablet_bootstrap.cc:492] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: Bootstrap starting.
I20260810 18:57:34.661216 27167 tablet_bootstrap.cc:654] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:34.660840 26609 tablet_service.cc:1467] Processing CreateTablet for tablet 0469b367e21e450b854b89e56f4c161c (DEFAULT_TABLE table=test-table [id=c37fea8d387a44fe953e3a3bceaa365e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:34.661433 27168 tablet_bootstrap.cc:654] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:34.662091 26609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0469b367e21e450b854b89e56f4c161c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:34.662377 27167 log.cc:824] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:34.662775 27168 log.cc:824] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:34.664824 27167 tablet_bootstrap.cc:492] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: No bootstrap required, opened a new log
I20260810 18:57:34.665047 27167 ts_tablet_manager.cc:1397] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:57:34.666157 27168 tablet_bootstrap.cc:492] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: No bootstrap required, opened a new log
I20260810 18:57:34.666335 27168 ts_tablet_manager.cc:1397] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260810 18:57:34.668797 27167 raft_consensus.cc:348] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.669030 27167 raft_consensus.cc:374] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:34.669103 27167 raft_consensus.cc:729] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2630a78187ad4accb4888d1b6534329d, State: Initialized, Role: FOLLOWER
I20260810 18:57:34.669589 27167 consensus_queue.cc:260] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.669726 27171 tablet_bootstrap.cc:492] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501: Bootstrap starting.
I20260810 18:57:34.670876 26928 heartbeater.cc:503] Master 127.30.153.190:33925 was elected leader, sending a full tablet report...
I20260810 18:57:34.671233 27167 ts_tablet_manager.cc:1428] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:57:34.670291 27168 raft_consensus.cc:348] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.671501 27168 raft_consensus.cc:374] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:34.671627 27168 raft_consensus.cc:729] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ffc0586db445a59d17392d6b61cd1e, State: Initialized, Role: FOLLOWER
I20260810 18:57:34.672067 27171 tablet_bootstrap.cc:654] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:34.672097 27168 consensus_queue.cc:260] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.672806 27134 heartbeater.cc:503] Master 127.30.153.190:33925 was elected leader, sending a full tablet report...
I20260810 18:57:34.672941 27171 log.cc:824] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:34.673074 27168 ts_tablet_manager.cc:1428] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260810 18:57:34.675474 27171 tablet_bootstrap.cc:492] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501: No bootstrap required, opened a new log
I20260810 18:57:34.675730 27171 ts_tablet_manager.cc:1397] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:57:34.679133 27171 raft_consensus.cc:348] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.679441 27171 raft_consensus.cc:374] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:34.679524 27171 raft_consensus.cc:729] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18729cbdb3db471a81c71f0d22417501, State: Initialized, Role: FOLLOWER
I20260810 18:57:34.679991 27171 consensus_queue.cc:260] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.680624 26716 heartbeater.cc:503] Master 127.30.153.190:33925 was elected leader, sending a full tablet report...
I20260810 18:57:34.680943 27171 ts_tablet_manager.cc:1428] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260810 18:57:34.698274 27178 raft_consensus.cc:482] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:34.698524 27178 raft_consensus.cc:504] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.699682 27178 leader_election.cc:288] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2630a78187ad4accb4888d1b6534329d (127.30.153.130:38835), e6ffc0586db445a59d17392d6b61cd1e (127.30.153.131:37941)
I20260810 18:57:34.705065 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.700012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:37325 (local address 127.30.153.130:38835)
0810 18:57:34.700274 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.700280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.700305 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:57:34.700500 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.700506 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.700572 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.700661 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:34.701004 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.701801 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.703625 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.704293 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.704411 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.704470 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.704823 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.704903 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260810 18:57:34.705065 27165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.700184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:49043 (local address 127.30.153.131:37941)
0810 18:57:34.700314 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.700319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.700517 (+ 198us) server_negotiation.cc:407] Connection header received
0810 18:57:34.700598 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.700603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.700664 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.700753 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:34.701004 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.701805 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.703625 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.704271 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.704411 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.704470 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.704823 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.704903 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:57:34.705150 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.699969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:37941 (local address 127.30.153.129:49043)
0810 18:57:34.700415 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.700430 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:34.700522 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:34.700773 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:34.700776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:34.700788 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:34.700893 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.700902 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.701932 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.701936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.703474 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.703488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.704433 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.704437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.704622 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.704703 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 18:57:34.704783 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 18:57:34.704915 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":263,"thread_start_us":130,"threads_started":1}
I20260810 18:57:34.705150 27180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.699885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:38835 (local address 127.30.153.129:37325)
0810 18:57:34.700220 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.700237 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:34.700356 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:34.700679 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:34.700681 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:34.700696 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:34.700858 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.700866 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.701932 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.701936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.703474 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:34.703488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.704433 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.704437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:34.704639 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.704697 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 18:57:34.704783 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 18:57:34.704921 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":112,"thread_start_us":139,"threads_started":1}
I20260810 18:57:34.705852 26848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0469b367e21e450b854b89e56f4c161c" candidate_uuid: "18729cbdb3db471a81c71f0d22417501" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2630a78187ad4accb4888d1b6534329d" is_pre_election: true
I20260810 18:57:34.706125 26848 raft_consensus.cc:2454] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18729cbdb3db471a81c71f0d22417501 in term 0.
I20260810 18:57:34.706660 26536 leader_election.cc:302] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18729cbdb3db471a81c71f0d22417501, 2630a78187ad4accb4888d1b6534329d; no voters:
I20260810 18:57:34.706692 27057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0469b367e21e450b854b89e56f4c161c" candidate_uuid: "18729cbdb3db471a81c71f0d22417501" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ffc0586db445a59d17392d6b61cd1e" is_pre_election: true
I20260810 18:57:34.706970 27057 raft_consensus.cc:2454] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18729cbdb3db471a81c71f0d22417501 in term 0.
I20260810 18:57:34.706990 27178 raft_consensus.cc:2781] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:57:34.707077 27178 raft_consensus.cc:482] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:57:34.707149 27178 raft_consensus.cc:3037] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:34.708384 27178 raft_consensus.cc:504] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.708791 27178 leader_election.cc:288] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [CANDIDATE]: Term 1 election: Requested vote from peers 2630a78187ad4accb4888d1b6534329d (127.30.153.130:38835), e6ffc0586db445a59d17392d6b61cd1e (127.30.153.131:37941)
I20260810 18:57:34.709143 26848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0469b367e21e450b854b89e56f4c161c" candidate_uuid: "18729cbdb3db471a81c71f0d22417501" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2630a78187ad4accb4888d1b6534329d"
I20260810 18:57:34.709282 26848 raft_consensus.cc:3037] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:34.710309 26848 raft_consensus.cc:2454] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18729cbdb3db471a81c71f0d22417501 in term 1.
I20260810 18:57:34.710610 27057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0469b367e21e450b854b89e56f4c161c" candidate_uuid: "18729cbdb3db471a81c71f0d22417501" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ffc0586db445a59d17392d6b61cd1e"
I20260810 18:57:34.710711 26536 leader_election.cc:302] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18729cbdb3db471a81c71f0d22417501, 2630a78187ad4accb4888d1b6534329d; no voters:
I20260810 18:57:34.710752 27057 raft_consensus.cc:3037] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:34.710870 27178 raft_consensus.cc:2781] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:34.711115 27178 raft_consensus.cc:686] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 1 LEADER]: Becoming Leader. State: Replica: 18729cbdb3db471a81c71f0d22417501, State: Running, Role: LEADER
I20260810 18:57:34.711474 27178 consensus_queue.cc:237] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } }
I20260810 18:57:34.712085 27057 raft_consensus.cc:2454] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18729cbdb3db471a81c71f0d22417501 in term 1.
I20260810 18:57:34.714629 26408 catalog_manager.cc:5495] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18729cbdb3db471a81c71f0d22417501 (127.30.153.129). New cstate: current_term: 1 leader_uuid: "18729cbdb3db471a81c71f0d22417501" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18729cbdb3db471a81c71f0d22417501" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 36117 } health_report { overall_health: HEALTHY } } }
I20260810 18:57:34.727577 27161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.723785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.30.153.129:36117)
0810 18:57:34.723898 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:34.723903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:34.724014 (+ 111us) server_negotiation.cc:407] Connection header received
0810 18:57:34.724184 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:34.724187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:34.724276 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:34.724395 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:57:34.724672 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.725181 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.726063 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:34.726290 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:34.726404 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:34.726745 (+ 341us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:57:34.727133 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:57:34.727305 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:34.727397 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 18:57:34.727438 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 18:57:34.733651 26848 raft_consensus.cc:1261] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 1 FOLLOWER]: Refusing update from remote peer 18729cbdb3db471a81c71f0d22417501: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:34.733650 27057 raft_consensus.cc:1261] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 1 FOLLOWER]: Refusing update from remote peer 18729cbdb3db471a81c71f0d22417501: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:34.734467 27182 consensus_queue.cc:1035] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ffc0586db445a59d17392d6b61cd1e" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 37941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:34.734717 27178 consensus_queue.cc:1035] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2630a78187ad4accb4888d1b6534329d" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 38835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:34.745249 27189 mvcc.cc:205] Tried to move back new op lower bound from 7317046291381350400 to 7317046291301281792. Current Snapshot: MvccSnapshot[applied={T|T < 7317046291381350400}]
I20260810 18:57:34.751924 26409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:34.752108 26409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53090:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:34.754519 26409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05b876a33eca4272815d66b6730f4720: Sending DeleteTablet for 3 replicas of tablet 0469b367e21e450b854b89e56f4c161c
I20260810 18:57:34.755311 27034 tablet_service.cc:1514] Processing DeleteTablet for tablet 0469b367e21e450b854b89e56f4c161c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:34 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:38428
I20260810 18:57:34.755326 26817 tablet_service.cc:1514] Processing DeleteTablet for tablet 0469b367e21e450b854b89e56f4c161c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:34 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:58388
I20260810 18:57:34.755559 26609 tablet_service.cc:1514] Processing DeleteTablet for tablet 0469b367e21e450b854b89e56f4c161c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:34 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:43022
I20260810 18:57:34.755832 27207 tablet_replica.cc:330] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: stopping tablet replica
I20260810 18:57:34.756009 27210 tablet_replica.cc:330] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501: stopping tablet replica
I20260810 18:57:34.756158 27207 raft_consensus.cc:2229] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:34.756341 27210 raft_consensus.cc:2229] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:57:34.756470 27207 raft_consensus.cc:2258] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:34.756727 27210 raft_consensus.cc:2258] T 0469b367e21e450b854b89e56f4c161c P 18729cbdb3db471a81c71f0d22417501 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:34.757000 27206 tablet_replica.cc:330] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: stopping tablet replica
I20260810 18:57:34.757099 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 26513
I20260810 18:57:34.757298 27206 raft_consensus.cc:2229] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:34.757565 27206 raft_consensus.cc:2258] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:34.758816 27207 ts_tablet_manager.cc:1905] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:34.761193 27206 ts_tablet_manager.cc:1905] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:34.762099 27207 ts_tablet_manager.cc:1918] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:34.762197 27207 log.cc:1196] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/0469b367e21e450b854b89e56f4c161c
I20260810 18:57:34.762604 27207 ts_tablet_manager.cc:1939] T 0469b367e21e450b854b89e56f4c161c P 2630a78187ad4accb4888d1b6534329d: Deleting consensus metadata
I20260810 18:57:34.764247 26381 catalog_manager.cc:4841] TS 2630a78187ad4accb4888d1b6534329d (127.30.153.130:38835): tablet 0469b367e21e450b854b89e56f4c161c (table test-table [id=c37fea8d387a44fe953e3a3bceaa365e]) successfully deleted
I20260810 18:57:34.764330 27206 ts_tablet_manager.cc:1918] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:34.764423 27206 log.cc:1196] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/0469b367e21e450b854b89e56f4c161c
I20260810 18:57:34.764780 27206 ts_tablet_manager.cc:1939] T 0469b367e21e450b854b89e56f4c161c P e6ffc0586db445a59d17392d6b61cd1e: Deleting consensus metadata
I20260810 18:57:34.765964 26379 catalog_manager.cc:4841] TS e6ffc0586db445a59d17392d6b61cd1e (127.30.153.131:37941): tablet 0469b367e21e450b854b89e56f4c161c (table test-table [id=c37fea8d387a44fe953e3a3bceaa365e]) successfully deleted
W20260810 18:57:34.772156 26378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:36117 (error 108)
I20260810 18:57:34.772475 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 26722
I20260810 18:57:34.775022 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:34.774646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:36117 (local address 127.0.0.1:43052)
0810 18:57:34.774842 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:34.774913 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260810 18:57:34.775177 26378 catalog_manager.cc:4584] TS 18729cbdb3db471a81c71f0d22417501 (127.30.153.129:36117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0469b367e21e450b854b89e56f4c161c: Network error: Client connection negotiation failed: client connection to 127.30.153.129:36117: connect: Connection refused (error 111)
I20260810 18:57:34.786041 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 26935
I20260810 18:57:34.796224 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 26351
2026-08-10T18:57:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2594 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T18:57:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:34Z Disabled control of system clock
I20260810 18:57:34.836278 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:41011
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:46495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:41011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:57:34.952292 27215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:34.952569 27215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:34.952606 27215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:34.956543 27215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:34.956599 27215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:34.956614 27215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:34.956631 27215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:34.956646 27215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:34.961328 27215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:41011
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:41011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:34.962502 27215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:34.963897 27215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:34.972661 27227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:34.972627 27230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:34.972611 27226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:34.972635 27215 server_base.cc:1034] running on GCE node
I20260810 18:57:34.973502 27215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:34.974630 27215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:34.975807 27215 hybrid_clock.cc:648] HybridClock initialized: now 1786388254975792 us; error 29 us; skew 500 ppm
I20260810 18:57:34.986788 27215 webserver.cc:460] Webserver started at http://127.30.153.190:37629/ using document root <none> and password file <none>
I20260810 18:57:34.987442 27215 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:34.987514 27215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:34.987772 27215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:34.989965 27215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "a1de183e850343c4ba96deb5fd4e56b4"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "37009206a319b4e5daf593913357e904"
server_key_iv: "b4d2db455cf7d95e3334cc8b321536cf"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.990613 27215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "a1de183e850343c4ba96deb5fd4e56b4"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "37009206a319b4e5daf593913357e904"
server_key_iv: "b4d2db455cf7d95e3334cc8b321536cf"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.994371 27215 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:34.996963 27237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:34.998250 27215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:34.998389 27215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "a1de183e850343c4ba96deb5fd4e56b4"
format_stamp: "Formatted at 2026-08-10 18:57:34 on dist-test-slave-68xk"
server_key: "37009206a319b4e5daf593913357e904"
server_key_iv: "b4d2db455cf7d95e3334cc8b321536cf"
server_key_version: "encryptionkey@0"
I20260810 18:57:34.998534 27215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:35.035789 27215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:35.036562 27215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:35.036798 27215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:35.044238 27215 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:41011
I20260810 18:57:35.044250 27324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:41011 every 8 connection(s)
I20260810 18:57:35.045264 27215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:35.048537 27325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:35.053251 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 27215
I20260810 18:57:35.053392 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:35.053714 31334 external_mini_cluster.cc:1428] Setting key 1d2ab82c89339ecff0dfb9bb197dc32e
I20260810 18:57:35.054958 27325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4: Bootstrap starting.
I20260810 18:57:35.057387 27325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:35.058266 27325 log.cc:824] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:35.060719 27325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4: No bootstrap required, opened a new log
I20260810 18:57:35.064067 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.055339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43348 (local address 127.30.153.190:41011)
0810 18:57:35.056020 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:35.056038 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:35.056076 (+ 38us) server_negotiation.cc:407] Connection header received
0810 18:57:35.056723 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:35.056739 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:35.057152 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:35.057443 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:35.057887 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.059216 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.060471 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.061045 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.061178 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.062103 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:35.062108 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:35.062121 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:35.062622 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:35.062683 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:35.062705 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:35.062809 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:35.063015 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 18:57:35.063335 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":146,"threads_started":1}
I20260810 18:57:35.065197 27325 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1de183e850343c4ba96deb5fd4e56b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41011 } }
I20260810 18:57:35.065425 27325 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:35.065501 27325 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1de183e850343c4ba96deb5fd4e56b4, State: Initialized, Role: FOLLOWER
I20260810 18:57:35.066000 27325 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1de183e850343c4ba96deb5fd4e56b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41011 } }
I20260810 18:57:35.066152 27325 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:35.066231 27325 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:35.066336 27325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:35.067327 27325 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1de183e850343c4ba96deb5fd4e56b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41011 } }
I20260810 18:57:35.067610 27325 leader_election.cc:302] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1de183e850343c4ba96deb5fd4e56b4; no voters:
I20260810 18:57:35.067874 27325 leader_election.cc:288] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:35.068109 27332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:35.068360 27332 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [term 1 LEADER]: Becoming Leader. State: Replica: a1de183e850343c4ba96deb5fd4e56b4, State: Running, Role: LEADER
I20260810 18:57:35.068693 27332 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1de183e850343c4ba96deb5fd4e56b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41011 } }
I20260810 18:57:35.069092 27325 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:35.070581 27336 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1de183e850343c4ba96deb5fd4e56b4. Latest consensus state: current_term: 1 leader_uuid: "a1de183e850343c4ba96deb5fd4e56b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1de183e850343c4ba96deb5fd4e56b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41011 } } }
I20260810 18:57:35.070557 27335 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1de183e850343c4ba96deb5fd4e56b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1de183e850343c4ba96deb5fd4e56b4" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 41011 } } }
I20260810 18:57:35.070732 27336 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:35.070770 27335 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4 [sys.catalog]: This master's current role is: LEADER
W20260810 18:57:35.074064 27359 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:57:35.074136 27359 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:57:35.074220 27347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:35.075021 27347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:35.079913 27347 catalog_manager.cc:1347] Generated new cluster ID: 01dc893367dd47fd8cfc730e8541233b
I20260810 18:57:35.079975 27347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:35.090492 27347 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:35.091776 27347 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:35.097539 27347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4: Generated new TSK 0
I20260810 18:57:35.098213 27347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:35.108309 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41011
--builtin_ntp_servers=127.30.153.148:46495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:57:35.230486 27364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:35.230746 27364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:35.230785 27364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:35.234844 27364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:35.234898 27364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:35.234951 27364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:57:35.239663 27364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:35.240715 27364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:35.242091 27364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:35.251889 27372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:35.251891 27373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:35.251935 27378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:35.252151 27364 server_base.cc:1034] running on GCE node
I20260810 18:57:35.252600 27364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:35.253294 27364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:35.254488 27364 hybrid_clock.cc:648] HybridClock initialized: now 1786388255254452 us; error 48 us; skew 500 ppm
I20260810 18:57:35.256265 27364 webserver.cc:460] Webserver started at http://127.30.153.129:36607/ using document root <none> and password file <none>
I20260810 18:57:35.256848 27364 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:35.256943 27364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:35.257184 27364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:35.259339 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "874b9ab13ac4421ba65282e98bdf2a64"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "5f9ae9f42ac5fcc9ac143d8f7b9e2fd0"
server_key_iv: "0dc9bbcc9dcb2994a858b485c3f4a9a6"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.259949 27364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "874b9ab13ac4421ba65282e98bdf2a64"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "5f9ae9f42ac5fcc9ac143d8f7b9e2fd0"
server_key_iv: "0dc9bbcc9dcb2994a858b485c3f4a9a6"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.263908 27364 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:35.266503 27385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.267735 27364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:35.267872 27364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "874b9ab13ac4421ba65282e98bdf2a64"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "5f9ae9f42ac5fcc9ac143d8f7b9e2fd0"
server_key_iv: "0dc9bbcc9dcb2994a858b485c3f4a9a6"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.267989 27364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:35.291492 27364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:35.292183 27364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:35.292402 27364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:35.293016 27364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:35.293989 27364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:35.294065 27364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.294137 27364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:35.294188 27364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.304145 27364 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:45689
I20260810 18:57:35.304178 27586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:45689 every 8 connection(s)
I20260810 18:57:35.305060 27364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:57:35.306867 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 27364
I20260810 18:57:35.307014 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:57:35.307358 31334 external_mini_cluster.cc:1428] Setting key 75b0c3de00efd6e3863e17a551b405fa
I20260810 18:57:35.309653 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41011
--builtin_ntp_servers=127.30.153.148:46495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:57:35.316288 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.307135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:43451 (local address 127.30.153.190:41011)
0810 18:57:35.307303 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:35.307307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:35.308558 (+ 1251us) server_negotiation.cc:407] Connection header received
0810 18:57:35.310073 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:35.310079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:35.310148 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:35.310246 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:35.311220 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.312058 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.313942 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.314186 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.314278 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.315765 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:35.315768 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:35.315771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:35.315784 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:35.315806 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:35.315816 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:35.315882 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:35.316108 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 18:57:35.316166 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 18:57:35.317615 27589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.307594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:41011 (local address 127.30.153.129:43451)
0810 18:57:35.308384 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:35.308437 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:35.309815 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:35.310384 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:35.310390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:35.310541 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:35.311059 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.311072 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.312212 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.312218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.313783 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.313799 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.314365 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.314371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.314670 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.315414 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:35.315417 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:35.315480 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:35.315578 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:35.315915 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:35.315929 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:35.315938 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:57:35.316096 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 18:57:35.316434 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":161,"threads_started":1}
I20260810 18:57:35.319092 27587 heartbeater.cc:347] Connected to a master server at 127.30.153.190:41011
I20260810 18:57:35.319422 27587 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:35.320011 27587 heartbeater.cc:511] Master 127.30.153.190:41011 requested a full tablet report, sending...
I20260810 18:57:35.321676 27260 ts_manager.cc:194] Registered new tserver with Master: 874b9ab13ac4421ba65282e98bdf2a64 (127.30.153.129:45689)
I20260810 18:57:35.323017 27260 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.129:43451
W20260810 18:57:35.428480 27590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:35.428758 27590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:35.428797 27590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:35.432880 27590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:35.432933 27590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:35.432987 27590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:57:35.437920 27590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:35.439322 27590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:35.441112 27590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:35.451404 27606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:35.451471 27590 server_base.cc:1034] running on GCE node
W20260810 18:57:35.451431 27604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:35.451404 27600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:35.452018 27590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:35.452680 27590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:35.453878 27590 hybrid_clock.cc:648] HybridClock initialized: now 1786388255453842 us; error 50 us; skew 500 ppm
I20260810 18:57:35.455752 27590 webserver.cc:460] Webserver started at http://127.30.153.130:42767/ using document root <none> and password file <none>
I20260810 18:57:35.456342 27590 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:35.456439 27590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:35.456677 27590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:35.458760 27590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "0c47f3d0603d47f79cc62281a860a226"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "c41365eba173870eb0afbad9a47e200b"
server_key_iv: "3a286ab0c58de543f4110c6f546c05c1"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.459280 27590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "0c47f3d0603d47f79cc62281a860a226"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "c41365eba173870eb0afbad9a47e200b"
server_key_iv: "3a286ab0c58de543f4110c6f546c05c1"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.462934 27590 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:57:35.465363 27618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.466638 27590 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:57:35.466743 27590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "0c47f3d0603d47f79cc62281a860a226"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "c41365eba173870eb0afbad9a47e200b"
server_key_iv: "3a286ab0c58de543f4110c6f546c05c1"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.466828 27590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:35.491235 27590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:35.492152 27590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:35.492381 27590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:35.493189 27590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:35.494630 27590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:35.494709 27590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.494786 27590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:35.494822 27590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.508502 27590 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:44611
I20260810 18:57:35.508522 27793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:44611 every 8 connection(s)
I20260810 18:57:35.509505 27590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:57:35.509769 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 27590
I20260810 18:57:35.509922 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:57:35.510305 31334 external_mini_cluster.cc:1428] Setting key ee394fc18b59ad249a8590f38e540a21
I20260810 18:57:35.512593 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41011
--builtin_ntp_servers=127.30.153.148:46495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:57:35.520380 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.511415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:45763 (local address 127.30.153.190:41011)
0810 18:57:35.511553 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:35.511557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:35.512937 (+ 1380us) server_negotiation.cc:407] Connection header received
0810 18:57:35.515006 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:35.515010 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:35.515065 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:35.515145 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:35.516045 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.516900 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.518053 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.518508 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.518663 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.519766 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:35.519771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:35.519777 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:35.519795 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:35.519825 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:35.519840 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:35.519930 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:35.520084 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 18:57:35.520163 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:57:35.521122 27796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.511852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:41011 (local address 127.30.153.130:45763)
0810 18:57:35.512763 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:35.512822 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:35.514787 (+ 1965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:35.515321 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:35.515328 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:35.515412 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:35.515893 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.515905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.517045 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.517048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.517927 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.517936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.518680 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.518683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.518835 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.519498 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:35.519500 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:35.519539 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:35.519600 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:35.519936 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:35.519943 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:35.519947 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:57:35.520044 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 18:57:35.520262 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":165,"threads_started":1}
I20260810 18:57:35.522284 27794 heartbeater.cc:347] Connected to a master server at 127.30.153.190:41011
I20260810 18:57:35.522569 27794 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:35.523088 27794 heartbeater.cc:511] Master 127.30.153.190:41011 requested a full tablet report, sending...
I20260810 18:57:35.524174 27260 ts_manager.cc:194] Registered new tserver with Master: 0c47f3d0603d47f79cc62281a860a226 (127.30.153.130:44611)
I20260810 18:57:35.524755 27260 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.130:45763
W20260810 18:57:35.634469 27798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:35.634750 27798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:35.634820 27798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:35.638792 27798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:35.638873 27798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:35.638962 27798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:57:35.643699 27798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:46495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:41011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:35.644833 27798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:35.646268 27798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:35.656409 27809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:35.656396 27804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:35.656400 27805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:35.656474 27798 server_base.cc:1034] running on GCE node
I20260810 18:57:35.657155 27798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:35.657860 27798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:35.659102 27798 hybrid_clock.cc:648] HybridClock initialized: now 1786388255659083 us; error 39 us; skew 500 ppm
I20260810 18:57:35.661300 27798 webserver.cc:460] Webserver started at http://127.30.153.131:36945/ using document root <none> and password file <none>
I20260810 18:57:35.661914 27798 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:35.662014 27798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:35.662271 27798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:35.664455 27798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "cbdb2fd291d24d65913b04a5d13415cc"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "6806f7f46e8b7573bee0e542915ef310"
server_key_iv: "664e88e45f1c3eac2fe019e1c7950b66"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.665045 27798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "cbdb2fd291d24d65913b04a5d13415cc"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "6806f7f46e8b7573bee0e542915ef310"
server_key_iv: "664e88e45f1c3eac2fe019e1c7950b66"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.670097 27798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 18:57:35.672974 27818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.674224 27798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:35.674366 27798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "cbdb2fd291d24d65913b04a5d13415cc"
format_stamp: "Formatted at 2026-08-10 18:57:35 on dist-test-slave-68xk"
server_key: "6806f7f46e8b7573bee0e542915ef310"
server_key_iv: "664e88e45f1c3eac2fe019e1c7950b66"
server_key_version: "encryptionkey@0"
I20260810 18:57:35.674525 27798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:35.699589 27798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:35.700320 27798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:35.700523 27798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:35.701187 27798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:35.702219 27798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:35.702294 27798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.702359 27798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:35.702411 27798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:35.711344 27798 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:39349
I20260810 18:57:35.711385 27982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:39349 every 8 connection(s)
I20260810 18:57:35.712244 27798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:57:35.721486 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 27798
I20260810 18:57:35.721683 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:57:35.722084 31334 external_mini_cluster.cc:1428] Setting key 422cddde44a15f5994cacf68bb74d93a
I20260810 18:57:35.722330 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.714122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:50711 (local address 127.30.153.190:41011)
0810 18:57:35.714293 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:35.714299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:35.715515 (+ 1216us) server_negotiation.cc:407] Connection header received
0810 18:57:35.717063 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:35.717067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:35.717135 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:35.717229 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:35.718242 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.718818 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.720084 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.720316 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.720403 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.721698 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:35.721702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:35.721705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:35.721718 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:35.721766 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:35.721781 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:35.721871 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:35.722072 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 18:57:35.722131 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 18:57:35.723556 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.714600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:41011 (local address 127.30.153.131:50711)
0810 18:57:35.715355 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:35.715401 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:35.716825 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:35.717415 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:35.717422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:35.717512 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:35.718030 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.718043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.718953 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.718956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.719958 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.719968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.720467 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.720470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.720627 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.721299 (+ 672us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:35.721302 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:35.721361 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:35.721452 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:35.721882 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:35.721896 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:35.721903 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 18:57:35.722057 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 18:57:35.722435 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":107,"threads_started":1}
I20260810 18:57:35.724814 27983 heartbeater.cc:347] Connected to a master server at 127.30.153.190:41011
I20260810 18:57:35.725083 27983 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:35.725693 27983 heartbeater.cc:511] Master 127.30.153.190:41011 requested a full tablet report, sending...
I20260810 18:57:35.726835 27260 ts_manager.cc:194] Registered new tserver with Master: cbdb2fd291d24d65913b04a5d13415cc (127.30.153.131:39349)
I20260810 18:57:35.727447 27260 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.131:50711
I20260810 18:57:35.736382 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:57:35.743026 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.738486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.30.153.190:41011)
0810 18:57:35.738669 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:35.738674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:35.738863 (+ 189us) server_negotiation.cc:407] Connection header received
0810 18:57:35.739027 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:35.739030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:35.739083 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:35.739185 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:35.739508 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.740077 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.741032 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.741278 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.741400 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.742378 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:35.742383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:35.742385 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:35.742397 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:35.742465 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:35.742478 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:35.742640 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:35.742817 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 18:57:35.742860 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:57:35.746270 27260 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:57:35.749614 27260 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:57:35.765378 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.759639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40882 (local address 127.30.153.131:39349)
0810 18:57:35.760036 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:35.760049 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:35.760489 (+ 440us) server_negotiation.cc:407] Connection header received
0810 18:57:35.760562 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:35.760567 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:35.760763 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:35.760910 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:35.761415 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.762243 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.763801 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.764606 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.764735 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.764913 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:35.765079 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 18:57:35.765192 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":165,"threads_started":1}
I20260810 18:57:35.765378 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.759026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:39349 (local address 127.0.0.1:40882)
0810 18:57:35.760161 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:35.760197 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:35.760340 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:35.761072 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:35.761078 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:35.761130 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:35.761273 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.761280 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.762369 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.762374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.763650 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.763661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.764741 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.764745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.764861 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.764909 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 18:57:35.765056 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 18:57:35.765217 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"spinlock_wait_cycles":7680,"thread_start_us":161,"threads_started":1}
I20260810 18:57:35.766145 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.758800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:44611 (local address 127.0.0.1:42390)
0810 18:57:35.760143 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:35.760197 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:35.760362 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:35.761070 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:35.761078 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:35.761136 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:35.761280 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.761287 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.763231 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.763237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.764371 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.764383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.765724 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.765728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.765821 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.765844 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 18:57:35.765929 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 18:57:35.765996 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"mutex_wait_us":4,"spinlock_wait_cycles":16896,"thread_start_us":87,"threads_started":1}
I20260810 18:57:35.766489 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.759231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45689 (local address 127.0.0.1:59266)
0810 18:57:35.760234 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:35.760251 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:35.760353 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:35.761266 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:35.761269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:35.761284 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:35.761379 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.761384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.762978 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.762982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.764123 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:35.764133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.766018 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.766022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:35.766127 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.766153 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:57:35.766246 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 18:57:35.766328 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":26,"thread_start_us":90,"threads_started":1}
I20260810 18:57:35.766918 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.758987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42390 (local address 127.30.153.130:44611)
0810 18:57:35.759489 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:35.759505 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:35.760300 (+ 795us) server_negotiation.cc:407] Connection header received
0810 18:57:35.760534 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:35.760540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:35.760763 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:35.760902 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:35.761437 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.762838 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.764515 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.765331 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.765465 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.765652 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:35.765971 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 18:57:35.766091 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":135,"threads_started":1}
I20260810 18:57:35.767235 28007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:35.759627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.30.153.129:45689)
0810 18:57:35.760734 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:35.760764 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:35.760795 (+ 31us) server_negotiation.cc:407] Connection header received
0810 18:57:35.760887 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:35.760893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:35.761106 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:35.761281 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:35.761489 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.762850 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.764263 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:35.765145 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:35.765270 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:35.765846 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:35.766755 (+ 909us) server_negotiation.cc:299] Negotiation successful
0810 18:57:35.767063 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":142,"threads_started":1}
I20260810 18:57:35.768895 27882 tablet_service.cc:1467] Processing CreateTablet for tablet 9889ecd5d59941c0b2998c70c09a3781 (DEFAULT_TABLE table=test-table [id=d98082c24a1e43aca839b805c91edbde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:35.770210 27882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9889ecd5d59941c0b2998c70c09a3781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:35.769757 27694 tablet_service.cc:1467] Processing CreateTablet for tablet 9889ecd5d59941c0b2998c70c09a3781 (DEFAULT_TABLE table=test-table [id=d98082c24a1e43aca839b805c91edbde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:35.769820 27476 tablet_service.cc:1467] Processing CreateTablet for tablet 9889ecd5d59941c0b2998c70c09a3781 (DEFAULT_TABLE table=test-table [id=d98082c24a1e43aca839b805c91edbde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:35.771046 27476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9889ecd5d59941c0b2998c70c09a3781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:35.771178 27694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9889ecd5d59941c0b2998c70c09a3781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:35.777758 28009 tablet_bootstrap.cc:492] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64: Bootstrap starting.
I20260810 18:57:35.779124 28013 tablet_bootstrap.cc:492] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Bootstrap starting.
I20260810 18:57:35.780336 28009 tablet_bootstrap.cc:654] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:35.780957 28013 tablet_bootstrap.cc:654] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:35.781235 28009 log.cc:824] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:35.781889 28013 log.cc:824] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:35.782261 28014 tablet_bootstrap.cc:492] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: Bootstrap starting.
I20260810 18:57:35.784013 28013 tablet_bootstrap.cc:492] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: No bootstrap required, opened a new log
I20260810 18:57:35.784245 28013 ts_tablet_manager.cc:1397] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 18:57:35.784436 28014 tablet_bootstrap.cc:654] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:35.785279 28014 log.cc:824] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:35.785599 28009 tablet_bootstrap.cc:492] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64: No bootstrap required, opened a new log
I20260810 18:57:35.785806 28009 ts_tablet_manager.cc:1397] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260810 18:57:35.787451 28014 tablet_bootstrap.cc:492] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: No bootstrap required, opened a new log
I20260810 18:57:35.787647 28014 ts_tablet_manager.cc:1397] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:57:35.789124 28013 raft_consensus.cc:348] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:35.789419 28013 raft_consensus.cc:374] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:35.789474 28013 raft_consensus.cc:729] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbdb2fd291d24d65913b04a5d13415cc, State: Initialized, Role: FOLLOWER
I20260810 18:57:35.789476 28009 raft_consensus.cc:348] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:35.789727 28009 raft_consensus.cc:374] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:35.789776 28009 raft_consensus.cc:729] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874b9ab13ac4421ba65282e98bdf2a64, State: Initialized, Role: FOLLOWER
I20260810 18:57:35.789995 28013 consensus_queue.cc:260] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:35.790262 28009 consensus_queue.cc:260] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:35.791281 28009 ts_tablet_manager.cc:1428] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260810 18:57:35.791316 28013 ts_tablet_manager.cc:1428] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260810 18:57:35.791435 27983 heartbeater.cc:503] Master 127.30.153.190:41011 was elected leader, sending a full tablet report...
I20260810 18:57:35.791586 27587 heartbeater.cc:503] Master 127.30.153.190:41011 was elected leader, sending a full tablet report...
I20260810 18:57:35.791975 28014 raft_consensus.cc:348] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:35.792148 28014 raft_consensus.cc:374] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:35.792179 28014 raft_consensus.cc:729] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c47f3d0603d47f79cc62281a860a226, State: Initialized, Role: FOLLOWER
I20260810 18:57:35.792572 28014 consensus_queue.cc:260] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:35.793247 27794 heartbeater.cc:503] Master 127.30.153.190:41011 was elected leader, sending a full tablet report...
I20260810 18:57:35.793464 28014 ts_tablet_manager.cc:1428] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260810 18:57:35.806182 27588 tablet.cc:2369] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:57:35.963274 27985 tablet.cc:2369] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 18:57:36.011093 27795 tablet.cc:2369] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:57:36.099725 28022 raft_consensus.cc:482] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:36.100020 28022 raft_consensus.cc:504] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:36.101415 28022 leader_election.cc:288] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbdb2fd291d24d65913b04a5d13415cc (127.30.153.131:39349), 874b9ab13ac4421ba65282e98bdf2a64 (127.30.153.129:45689)
I20260810 18:57:36.105872 28008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:36.101787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:36649 (local address 127.30.153.131:39349)
0810 18:57:36.102002 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:36.102007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:36.102332 (+ 325us) server_negotiation.cc:407] Connection header received
0810 18:57:36.102801 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:36.102803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:36.102875 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:36.102975 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:36.103295 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.103785 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.104823 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.105233 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.105363 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:36.105415 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:36.105661 (+ 246us) server_negotiation.cc:299] Negotiation successful
0810 18:57:36.105736 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 18:57:36.105901 28007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:36.101807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:41003 (local address 127.30.153.129:45689)
0810 18:57:36.102014 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:36.102021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:36.102294 (+ 273us) server_negotiation.cc:407] Connection header received
0810 18:57:36.102494 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:36.102497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:36.102579 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:36.102671 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:36.102986 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.103603 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.104789 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.105223 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.105375 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:36.105427 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:36.105669 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 18:57:36.105744 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260810 18:57:36.105964 28025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:36.101687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45689 (local address 127.30.153.130:41003)
0810 18:57:36.102182 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:36.102200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:36.102353 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:36.102707 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:36.102709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:36.102729 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:36.102839 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:36.102844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.103728 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.103735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:36.104635 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:36.104646 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.105380 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.105384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:36.105514 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:36.105548 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 18:57:36.105646 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 18:57:36.105732 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":9,"spinlock_wait_cycles":20096,"thread_start_us":145,"threads_started":1}
I20260810 18:57:36.106107 28026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:36.101691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:39349 (local address 127.30.153.130:36649)
0810 18:57:36.102182 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:36.102200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:36.102353 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:36.102990 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:36.102993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:36.103013 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:36.103146 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:36.103151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.103899 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.103905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:36.104682 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:36.104691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.105357 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.105361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:36.105458 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:36.105551 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 18:57:36.105642 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 18:57:36.105728 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":138,"spinlock_wait_cycles":16000,"thread_start_us":124,"threads_started":1}
I20260810 18:57:36.106693 27509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9889ecd5d59941c0b2998c70c09a3781" candidate_uuid: "0c47f3d0603d47f79cc62281a860a226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874b9ab13ac4421ba65282e98bdf2a64" is_pre_election: true
I20260810 18:57:36.106830 27908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9889ecd5d59941c0b2998c70c09a3781" candidate_uuid: "0c47f3d0603d47f79cc62281a860a226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbdb2fd291d24d65913b04a5d13415cc" is_pre_election: true
I20260810 18:57:36.107048 27509 raft_consensus.cc:2454] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c47f3d0603d47f79cc62281a860a226 in term 0.
I20260810 18:57:36.107123 27908 raft_consensus.cc:2454] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c47f3d0603d47f79cc62281a860a226 in term 0.
I20260810 18:57:36.107607 27623 leader_election.cc:302] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c47f3d0603d47f79cc62281a860a226, 874b9ab13ac4421ba65282e98bdf2a64; no voters:
I20260810 18:57:36.107884 28022 raft_consensus.cc:2781] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:57:36.107980 28022 raft_consensus.cc:482] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:57:36.108028 28022 raft_consensus.cc:3037] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:36.109133 28022 raft_consensus.cc:504] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:36.109483 28022 leader_election.cc:288] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [CANDIDATE]: Term 1 election: Requested vote from peers cbdb2fd291d24d65913b04a5d13415cc (127.30.153.131:39349), 874b9ab13ac4421ba65282e98bdf2a64 (127.30.153.129:45689)
I20260810 18:57:36.109754 27908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9889ecd5d59941c0b2998c70c09a3781" candidate_uuid: "0c47f3d0603d47f79cc62281a860a226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbdb2fd291d24d65913b04a5d13415cc"
I20260810 18:57:36.109879 27509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9889ecd5d59941c0b2998c70c09a3781" candidate_uuid: "0c47f3d0603d47f79cc62281a860a226" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874b9ab13ac4421ba65282e98bdf2a64"
I20260810 18:57:36.109900 27908 raft_consensus.cc:3037] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:36.110010 27509 raft_consensus.cc:3037] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:36.111268 27908 raft_consensus.cc:2454] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c47f3d0603d47f79cc62281a860a226 in term 1.
I20260810 18:57:36.111384 27509 raft_consensus.cc:2454] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c47f3d0603d47f79cc62281a860a226 in term 1.
I20260810 18:57:36.111656 27624 leader_election.cc:302] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c47f3d0603d47f79cc62281a860a226, cbdb2fd291d24d65913b04a5d13415cc; no voters:
I20260810 18:57:36.111864 28022 raft_consensus.cc:2781] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:36.112092 28022 raft_consensus.cc:686] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 1 LEADER]: Becoming Leader. State: Replica: 0c47f3d0603d47f79cc62281a860a226, State: Running, Role: LEADER
I20260810 18:57:36.112452 28022 consensus_queue.cc:237] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } }
I20260810 18:57:36.115886 27258 catalog_manager.cc:5495] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c47f3d0603d47f79cc62281a860a226 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "0c47f3d0603d47f79cc62281a860a226" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c47f3d0603d47f79cc62281a860a226" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 44611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 } health_report { overall_health: UNKNOWN } } }
I20260810 18:57:36.130744 28001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:36.127443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42396 (local address 127.30.153.130:44611)
0810 18:57:36.127571 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:36.127575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:36.127589 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:57:36.127653 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:36.127655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:36.127706 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:36.127800 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:57:36.128040 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.128529 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.129365 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:36.129579 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:36.129663 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:36.129944 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:57:36.130355 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:57:36.130536 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:36.130608 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 18:57:36.130637 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:57:36.136695 27908 raft_consensus.cc:1261] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 1 FOLLOWER]: Refusing update from remote peer 0c47f3d0603d47f79cc62281a860a226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:36.136695 27509 raft_consensus.cc:1261] T 9889ecd5d59941c0b2998c70c09a3781 P 874b9ab13ac4421ba65282e98bdf2a64 [term 1 FOLLOWER]: Refusing update from remote peer 0c47f3d0603d47f79cc62281a860a226: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:36.137539 28022 consensus_queue.cc:1035] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbdb2fd291d24d65913b04a5d13415cc" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 39349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:36.137785 28027 consensus_queue.cc:1035] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [LEADER]: Connected to new peer: Peer: permanent_uuid: "874b9ab13ac4421ba65282e98bdf2a64" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:36.147418 28034 mvcc.cc:205] Tried to move back new op lower bound from 7317046297128042496 to 7317046297039208448. Current Snapshot: MvccSnapshot[applied={T|T < 7317046297128042496}]
I20260810 18:57:36.149607 28033 mvcc.cc:205] Tried to move back new op lower bound from 7317046297128042496 to 7317046297039208448. Current Snapshot: MvccSnapshot[applied={T|T < 7317046297128042496 or (T in {7317046297128042496})}]
I20260810 18:57:36.155766 27258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:36.155923 27258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:36.158363 27258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1de183e850343c4ba96deb5fd4e56b4: Sending DeleteTablet for 3 replicas of tablet 9889ecd5d59941c0b2998c70c09a3781
I20260810 18:57:36.159199 27694 tablet_service.cc:1514] Processing DeleteTablet for tablet 9889ecd5d59941c0b2998c70c09a3781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:36 UTC) from {username='slave'} at 127.0.0.1:42390
I20260810 18:57:36.159353 27476 tablet_service.cc:1514] Processing DeleteTablet for tablet 9889ecd5d59941c0b2998c70c09a3781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:36 UTC) from {username='slave'} at 127.0.0.1:59266
I20260810 18:57:36.159708 28045 tablet_replica.cc:330] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: stopping tablet replica
I20260810 18:57:36.159744 27882 tablet_service.cc:1514] Processing DeleteTablet for tablet 9889ecd5d59941c0b2998c70c09a3781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:36 UTC) from {username='slave'} at 127.0.0.1:40882
I20260810 18:57:36.160041 28045 raft_consensus.cc:2229] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:57:36.160308 28050 tablet_replica.cc:330] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: stopping tablet replica
I20260810 18:57:36.160423 28045 raft_consensus.cc:2258] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:36.160722 28050 raft_consensus.cc:2229] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:36.161007 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 27364
I20260810 18:57:36.169574 28050 raft_consensus.cc:2258] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:36.173501 28050 ts_tablet_manager.cc:1905] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:36.176712 28050 ts_tablet_manager.cc:1918] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:36.176877 28050 log.cc:1196] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/9889ecd5d59941c0b2998c70c09a3781
I20260810 18:57:36.177301 28050 ts_tablet_manager.cc:1939] T 9889ecd5d59941c0b2998c70c09a3781 P cbdb2fd291d24d65913b04a5d13415cc: Deleting consensus metadata
I20260810 18:57:36.178965 27241 catalog_manager.cc:4841] TS cbdb2fd291d24d65913b04a5d13415cc (127.30.153.131:39349): tablet 9889ecd5d59941c0b2998c70c09a3781 (table test-table [id=d98082c24a1e43aca839b805c91edbde]) successfully deleted
I20260810 18:57:36.183885 28045 ts_tablet_manager.cc:1905] T 9889ecd5d59941c0b2998c70c09a3781 P 0c47f3d0603d47f79cc62281a860a226: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 18:57:36.184126 27240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:45689 (error 108)
I20260810 18:57:36.185217 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:36.184870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45689 (local address 127.0.0.1:59276)
0810 18:57:36.185006 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:36.185101 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 18:57:36.185365 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 27590
W20260810 18:57:36.189009 27240 catalog_manager.cc:4584] TS 874b9ab13ac4421ba65282e98bdf2a64 (127.30.153.129:45689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9889ecd5d59941c0b2998c70c09a3781: Network error: Client connection negotiation failed: client connection to 127.30.153.129:45689: connect: Connection refused (error 111)
I20260810 18:57:36.195708 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 27798
I20260810 18:57:36.211601 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:36.195969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:44611 (local address 127.0.0.1:42400)
0810 18:57:36.211373 (+ 15404us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:36.211472 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.130:44611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15260}
I20260810 18:57:36.218101 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 27215
2026-08-10T18:57:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1422 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:36 dist-test-slave-68xk krb5kdc[28057](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:36 dist-test-slave-68xk krb5kdc[28057](info): set up 2 sockets
Aug 10 18:57:36 dist-test-slave-68xk krb5kdc[28057](info): commencing operation
krb5kdc: starting...
W20260810 18:57:37.513847 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:37 dist-test-slave-68xk krb5kdc[28057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:57:37.653407 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42511
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:42511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:37.772990 28072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:37.773401 28072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:37.773500 28072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:37.778224 28072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:37.778326 28072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:37.778369 28072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:37.778465 28072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:37.778532 28072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:37.784618 28072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:42511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:37.785848 28072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:37.787060 28072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:37.796687 28084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:37.796690 28082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:37.796687 28086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:37.796959 28072 server_base.cc:1034] running on GCE node
I20260810 18:57:37.797679 28072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:37.798882 28072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:37.800138 28072 hybrid_clock.cc:648] HybridClock initialized: now 1786388257800070 us; error 42 us; skew 500 ppm
Aug 10 18:57:37 dist-test-slave-68xk krb5kdc[28057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388257, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:37.803133 28072 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:57:37.804019 28072 webserver.cc:460] Webserver started at http://127.30.153.190:39369/ using document root <none> and password file <none>
I20260810 18:57:37.804641 28072 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:37.804740 28072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:37.804977 28072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:37.807148 28072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "0fe88820bf3f44bba0e13e23518da547"
format_stamp: "Formatted at 2026-08-10 18:57:37 on dist-test-slave-68xk"
server_key: "6de1a139eec0ae5c5be6935ed742f8d9"
server_key_iv: "01b41f402a49c4a8f5a4a47c501840da"
server_key_version: "encryptionkey@0"
I20260810 18:57:37.807731 28072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "0fe88820bf3f44bba0e13e23518da547"
format_stamp: "Formatted at 2026-08-10 18:57:37 on dist-test-slave-68xk"
server_key: "6de1a139eec0ae5c5be6935ed742f8d9"
server_key_iv: "01b41f402a49c4a8f5a4a47c501840da"
server_key_version: "encryptionkey@0"
I20260810 18:57:37.811560 28072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:57:37.813988 28098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:37.815482 28072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:37.815656 28072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "0fe88820bf3f44bba0e13e23518da547"
format_stamp: "Formatted at 2026-08-10 18:57:37 on dist-test-slave-68xk"
server_key: "6de1a139eec0ae5c5be6935ed742f8d9"
server_key_iv: "01b41f402a49c4a8f5a4a47c501840da"
server_key_version: "encryptionkey@0"
I20260810 18:57:37.815781 28072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:37.836753 28072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:37.840154 28072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:37.840375 28072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:37.847925 28072 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:42511
I20260810 18:57:37.847946 28184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:42511 every 8 connection(s)
I20260810 18:57:37.848984 28072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:37.851106 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 28072
I20260810 18:57:37.851264 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:37.851552 31334 external_mini_cluster.cc:1428] Setting key 47cb8b13c4ea847671ccb974fd68d2f3
I20260810 18:57:37.852100 28185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:37.858532 28185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547: Bootstrap starting.
Aug 10 18:57:37 dist-test-slave-68xk krb5kdc[28057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:37.861418 28185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:37.862515 28185 log.cc:824] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:37.864972 28185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547: No bootstrap required, opened a new log
I20260810 18:57:37.867239 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:37.852967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44532 (local address 127.30.153.190:42511)
0810 18:57:37.853443 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:37.853458 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:37.853492 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:57:37.854125 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:37.854141 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:37.854536 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:37.854873 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:37.855712 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:37.856704 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:37.857598 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:37.858072 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:37.858256 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:37.861157 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:37.861163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:37.861180 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:37.861217 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:37.863387 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:37.863849 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:37.863855 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:37.863862 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:37.864012 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:37.864365 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:37.864369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:37.864371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:37.864933 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:37.865178 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:37.865537 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 18:57:37.865871 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":169,"threads_started":1}
I20260810 18:57:37.869228 28185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe88820bf3f44bba0e13e23518da547" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42511 } }
I20260810 18:57:37.869503 28185 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:37.869619 28185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe88820bf3f44bba0e13e23518da547, State: Initialized, Role: FOLLOWER
I20260810 18:57:37.870108 28185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe88820bf3f44bba0e13e23518da547" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42511 } }
I20260810 18:57:37.870244 28185 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:37.870288 28185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:37.870365 28185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:37.871320 28185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe88820bf3f44bba0e13e23518da547" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42511 } }
I20260810 18:57:37.871634 28185 leader_election.cc:302] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fe88820bf3f44bba0e13e23518da547; no voters:
I20260810 18:57:37.871978 28185 leader_election.cc:288] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:37.872182 28194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:37.872665 28194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [term 1 LEADER]: Becoming Leader. State: Replica: 0fe88820bf3f44bba0e13e23518da547, State: Running, Role: LEADER
I20260810 18:57:37.872997 28194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe88820bf3f44bba0e13e23518da547" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42511 } }
I20260810 18:57:37.873487 28185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:37.874701 28195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fe88820bf3f44bba0e13e23518da547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe88820bf3f44bba0e13e23518da547" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42511 } } }
I20260810 18:57:37.874867 28195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:37.875278 28205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:37.875298 28196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fe88820bf3f44bba0e13e23518da547. Latest consensus state: current_term: 1 leader_uuid: "0fe88820bf3f44bba0e13e23518da547" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe88820bf3f44bba0e13e23518da547" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42511 } } }
I20260810 18:57:37.875946 28196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:37.877969 28205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:37.883623 28205 catalog_manager.cc:1347] Generated new cluster ID: d6858840bdb24c0c8361eb749847d89d
I20260810 18:57:37.883694 28205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:37.894716 28205 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:37.896088 28205 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:37.908681 28205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547: Generated new TSK 0
I20260810 18:57:37.909374 28205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.153.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
Entry for principal kudu/127.30.153.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab.
I20260810 18:57:37.949254 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42511
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:38.092960 28228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:38.093210 28228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:38.093253 28228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:38.097285 28228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:38.097347 28228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:38.097429 28228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:57:38.102665 28228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.129.keytab
--principal=kudu/127.30.153.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:38.103703 28228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:38.104755 28228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:38.114593 28235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:38.114578 28238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:38.114617 28228 server_base.cc:1034] running on GCE node
W20260810 18:57:38.114557 28236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:38.115290 28228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:38.115942 28228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:38.117149 28228 hybrid_clock.cc:648] HybridClock initialized: now 1786388258117115 us; error 47 us; skew 500 ppm
Aug 10 18:57:38 dist-test-slave-68xk krb5kdc[28057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:38.120671 28228 init.cc:377] Logged in from keytab as kudu/127.30.153.129@KRBTEST.COM (short username kudu)
I20260810 18:57:38.121685 28228 webserver.cc:460] Webserver started at http://127.30.153.129:37399/ using document root <none> and password file <none>
I20260810 18:57:38.122289 28228 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:38.122362 28228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:38.122659 28228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:38.124774 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "f175a5f15f994f0aa1ee0cedcf9d319f"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "4d50cc32105e1273bbd6cef6cb65ffde"
server_key_iv: "61515cd79f03051a5b5b6d03fcbb4b23"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.125383 28228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "f175a5f15f994f0aa1ee0cedcf9d319f"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "4d50cc32105e1273bbd6cef6cb65ffde"
server_key_iv: "61515cd79f03051a5b5b6d03fcbb4b23"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.129313 28228 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260810 18:57:38.131804 28250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.133033 28228 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:57:38.133232 28228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "f175a5f15f994f0aa1ee0cedcf9d319f"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "4d50cc32105e1273bbd6cef6cb65ffde"
server_key_iv: "61515cd79f03051a5b5b6d03fcbb4b23"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.133358 28228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:38.155652 28228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:38.159569 28228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:38.159806 28228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:38.160511 28228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:38.161475 28228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:38.161561 28228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.161638 28228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:38.161693 28228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.172016 28228 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:42377
I20260810 18:57:38.172049 28437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:42377 every 8 connection(s)
I20260810 18:57:38.173024 28228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:57:38.178654 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 28228
I20260810 18:57:38.178812 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:57:38.179170 31334 external_mini_cluster.cc:1428] Setting key 677ae6183a74385991fce4dce14fd5f4
Aug 10 18:57:38 dist-test-slave-68xk krb5kdc[28057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.129@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:38.189178 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.175256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:36411 (local address 127.30.153.190:42511)
0810 18:57:38.175466 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.175470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.176352 (+ 882us) server_negotiation.cc:407] Connection header received
0810 18:57:38.177709 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.177713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.177770 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.177880 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:38.179250 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.179946 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.181237 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.181492 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.181606 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.184717 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:38.184724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:38.184730 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:38.184764 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:38.186790 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:38.187352 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:38.187359 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:38.187364 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:38.187468 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:38.187855 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:38.187861 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:38.187865 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:38.188147 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:38.188286 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.188808 (+ 522us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.188958 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 18:57:38.189915 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.175704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42511 (local address 127.30.153.129:36411)
0810 18:57:38.176183 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.176227 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.177495 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.178072 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.178080 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.178516 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:38.179096 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.179111 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.180082 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.180085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.181104 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.181115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.181657 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.181662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.181829 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.182517 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:38.182534 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:38.184504 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:38.186945 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:38.186950 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:38.186961 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:38.187230 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:38.187576 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:38.187578 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:38.187580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:38.187746 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:38.188298 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:38.188303 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:38.188540 (+ 237us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.188721 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.188979 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.30.153.130@KRBTEST.COM; defaulting to no policy
I20260810 18:57:38.191310 28439 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42511
I20260810 18:57:38.191582 28439 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:38.192077 28439 heartbeater.cc:511] Master 127.30.153.190:42511 requested a full tablet report, sending...
I20260810 18:57:38.193747 28137 ts_manager.cc:194] Registered new tserver with Master: f175a5f15f994f0aa1ee0cedcf9d319f (127.30.153.129:42377)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.130@KRBTEST.COM" created.
I20260810 18:57:38.195086 28137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.129@KRBTEST.COM'} at 127.30.153.129:36411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
Entry for principal kudu/127.30.153.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab.
I20260810 18:57:38.209868 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42511
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:38.325006 28446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:38.325323 28446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:38.325371 28446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:38.329607 28446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:38.329681 28446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:38.329774 28446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:57:38.335239 28446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.130.keytab
--principal=kudu/127.30.153.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:38.336535 28446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:38.337718 28446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:38.348142 28458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:38.348094 28454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:38.348057 28452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:38.348439 28446 server_base.cc:1034] running on GCE node
I20260810 18:57:38.348870 28446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:38.349587 28446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:38.350793 28446 hybrid_clock.cc:648] HybridClock initialized: now 1786388258350770 us; error 33 us; skew 500 ppm
Aug 10 18:57:38 dist-test-slave-68xk krb5kdc[28057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:38.353914 28446 init.cc:377] Logged in from keytab as kudu/127.30.153.130@KRBTEST.COM (short username kudu)
I20260810 18:57:38.354880 28446 webserver.cc:460] Webserver started at http://127.30.153.130:42515/ using document root <none> and password file <none>
I20260810 18:57:38.355484 28446 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:38.355580 28446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:38.355829 28446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:38.357942 28446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "ff2ace79c463416c917c7917425f4bab"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "3d5f19a87e1319c74ebffdfc7b1941d8"
server_key_iv: "bcccbaac0ae3f72ab24a3edaa360eef6"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.358599 28446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "ff2ace79c463416c917c7917425f4bab"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "3d5f19a87e1319c74ebffdfc7b1941d8"
server_key_iv: "bcccbaac0ae3f72ab24a3edaa360eef6"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.362481 28446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:38.364939 28469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.366185 28446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:38.366348 28446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "ff2ace79c463416c917c7917425f4bab"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "3d5f19a87e1319c74ebffdfc7b1941d8"
server_key_iv: "bcccbaac0ae3f72ab24a3edaa360eef6"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.366524 28446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:38.399034 28446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:38.402886 28446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:38.403117 28446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:38.403759 28446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:38.404635 28446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:38.404711 28446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.404784 28446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:38.404829 28446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.414288 28446 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:32859
I20260810 18:57:38.414332 28653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:32859 every 8 connection(s)
I20260810 18:57:38.415259 28446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:57:38.417251 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 28446
I20260810 18:57:38.417399 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:57:38.417701 31334 external_mini_cluster.cc:1428] Setting key 17753382543933ed6495d7d651336bf2
Aug 10 18:57:38 dist-test-slave-68xk krb5kdc[28057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.130@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.131@KRBTEST.COM; defaulting to no policy
I20260810 18:57:38.431790 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.417055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:41103 (local address 127.30.153.190:42511)
0810 18:57:38.417231 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.417235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.418093 (+ 858us) server_negotiation.cc:407] Connection header received
0810 18:57:38.419816 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.419820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.419883 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.419957 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:38.421387 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.422268 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.423645 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.424063 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.424198 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.427524 (+ 3326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:38.427528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:38.427533 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:38.427569 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:38.429436 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:38.430080 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:38.430086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:38.430091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:38.430197 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:38.430602 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:38.430605 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:38.430607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:38.430818 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:38.430947 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.431529 (+ 582us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.431662 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 18:57:38.432741 28657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.417427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42511 (local address 127.30.153.130:41103)
0810 18:57:38.417937 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.417991 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.419562 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.420310 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.420317 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.420749 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:38.421240 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.421254 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.422507 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.422510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.423479 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.423490 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.424316 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.424321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.424526 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.425310 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:38.425328 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:38.427314 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:38.429580 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:38.429585 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:38.429599 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:38.429951 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:38.430294 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:38.430297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:38.430298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:38.430503 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:38.430979 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:38.430988 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:38.431284 (+ 296us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.431448 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.431745 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":148,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.131@KRBTEST.COM" created.
I20260810 18:57:38.434074 28655 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42511
I20260810 18:57:38.434351 28655 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:38.434906 28655 heartbeater.cc:511] Master 127.30.153.190:42511 requested a full tablet report, sending...
I20260810 18:57:38.436128 28137 ts_manager.cc:194] Registered new tserver with Master: ff2ace79c463416c917c7917425f4bab (127.30.153.130:32859)
I20260810 18:57:38.436856 28137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.130@KRBTEST.COM'} at 127.30.153.130:41103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
Entry for principal kudu/127.30.153.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab.
I20260810 18:57:38.450261 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42511
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:33707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:38.565719 28663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:38.565959 28663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:38.566001 28663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:38.570035 28663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:38.570091 28663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:38.570192 28663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:57:38.575676 28663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:33707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.131.keytab
--principal=kudu/127.30.153.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:42511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:38.576653 28663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:38.577659 28663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:38.587323 28675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:38.587311 28670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:38.587306 28673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:38.587646 28663 server_base.cc:1034] running on GCE node
I20260810 18:57:38.588163 28663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:38.588786 28663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:38.589996 28663 hybrid_clock.cc:648] HybridClock initialized: now 1786388258589971 us; error 38 us; skew 500 ppm
Aug 10 18:57:38 dist-test-slave-68xk krb5kdc[28057](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:38.592901 28663 init.cc:377] Logged in from keytab as kudu/127.30.153.131@KRBTEST.COM (short username kudu)
I20260810 18:57:38.593855 28663 webserver.cc:460] Webserver started at http://127.30.153.131:42207/ using document root <none> and password file <none>
I20260810 18:57:38.594388 28663 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:38.594519 28663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:38.594710 28663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:38.596710 28663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "87a3230760e045c4a8d36b8e2f216e1d"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "0f072a343f9f3b3975818a7ce4b41bd0"
server_key_iv: "b111dcb9377e3c82a915813ee787f081"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.597242 28663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "87a3230760e045c4a8d36b8e2f216e1d"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "0f072a343f9f3b3975818a7ce4b41bd0"
server_key_iv: "b111dcb9377e3c82a915813ee787f081"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.601064 28663 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:57:38.603832 28690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.605441 28663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:57:38.605582 28663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "87a3230760e045c4a8d36b8e2f216e1d"
format_stamp: "Formatted at 2026-08-10 18:57:38 on dist-test-slave-68xk"
server_key: "0f072a343f9f3b3975818a7ce4b41bd0"
server_key_iv: "b111dcb9377e3c82a915813ee787f081"
server_key_version: "encryptionkey@0"
I20260810 18:57:38.605706 28663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:38.627640 28663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:38.630877 28663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:38.631089 28663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:38.631717 28663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:38.632570 28663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:38.632643 28663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.632711 28663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:38.632756 28663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:38.642184 28663 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:41261
I20260810 18:57:38.642207 28869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:41261 every 8 connection(s)
I20260810 18:57:38.643115 28663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:57:38.648226 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 28663
I20260810 18:57:38.648396 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:57:38.648697 31334 external_mini_cluster.cc:1428] Setting key 252d001e15b511135faba056ce9e31fa
Aug 10 18:57:38 dist-test-slave-68xk krb5kdc[28057](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388258, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.131@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:57:38.657362 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.645120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:45059 (local address 127.30.153.190:42511)
0810 18:57:38.645309 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.645314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.646232 (+ 918us) server_negotiation.cc:407] Connection header received
0810 18:57:38.647541 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.647544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.647599 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.647673 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:38.648974 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.649521 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.651012 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.651242 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.651355 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.653747 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:38.653753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:38.653761 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:38.653783 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:38.655218 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:38.655735 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:38.655740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:38.655742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:38.655829 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:38.656176 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:38.656182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:38.656187 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:38.656408 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:38.656525 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.657007 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.657128 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 18:57:38.658128 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.645573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:42511 (local address 127.30.153.131:45059)
0810 18:57:38.646043 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.646087 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.647345 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.647841 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.647847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.648240 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:38.648824 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.648840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.649669 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.649673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.650884 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.650902 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.651391 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.651395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.651575 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.652122 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:57:38.652138 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:38.653591 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:38.655346 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:38.655352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:38.655366 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:38.655624 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:38.655929 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:57:38.655932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:57:38.655934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:57:38.656074 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:57:38.656525 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:38.656532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:38.656776 (+ 244us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.656963 (+ 187us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.657283 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":120,"threads_started":1}
I20260810 18:57:38.659209 28870 heartbeater.cc:347] Connected to a master server at 127.30.153.190:42511
I20260810 18:57:38.659446 28870 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:38.659906 28870 heartbeater.cc:511] Master 127.30.153.190:42511 requested a full tablet report, sending...
I20260810 18:57:38.660918 28137 ts_manager.cc:194] Registered new tserver with Master: 87a3230760e045c4a8d36b8e2f216e1d (127.30.153.131:41261)
I20260810 18:57:38.661690 28137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.153.131@KRBTEST.COM'} at 127.30.153.131:45059
I20260810 18:57:38.662726 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:57:38.672099 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.664710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44536 (local address 127.30.153.190:42511)
0810 18:57:38.664834 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.664838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.665078 (+ 240us) server_negotiation.cc:407] Connection header received
0810 18:57:38.665195 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.665197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.665237 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.665297 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:38.665928 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.666499 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.667342 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.667546 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.667624 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.669094 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:38.669097 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:38.669099 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:38.669119 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:38.670497 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:38.671030 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:38.671033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:38.671034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:38.671107 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:38.671373 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:38.671375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:38.671376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:38.671568 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:38.671674 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.671903 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.671994 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 18:57:38.675031 28137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:57:38.678458 28137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:57:38.694205 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.688355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:42377 (local address 127.0.0.1:39440)
0810 18:57:38.689817 (+ 1462us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.689830 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.689975 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.690507 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.690512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.690537 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:38.690655 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.690662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.691968 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.691971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.692784 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.692794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.693616 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.693619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.693737 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.693775 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.693911 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.694033 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1407,"thread_start_us":103,"threads_started":1}
I20260810 18:57:38.694207 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.688896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39440 (local address 127.30.153.129:42377)
0810 18:57:38.689656 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.689695 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.689913 (+ 218us) server_negotiation.cc:407] Connection header received
0810 18:57:38.690141 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.690147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.690347 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.690527 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:38.690791 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.691854 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.692913 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.693487 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.693629 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.693770 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.693945 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.694050 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":101,"threads_started":1}
I20260810 18:57:38.694816 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.688979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.30.153.131:41261)
0810 18:57:38.689443 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.689454 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.689475 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:57:38.689558 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.689563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.689729 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.690194 (+ 465us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:38.690335 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.691546 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.692848 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.693683 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.694371 (+ 688us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.694519 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.694568 (+ 49us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.694657 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":167,"threads_started":1}
I20260810 18:57:38.694244 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.688069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:41261 (local address 127.0.0.1:35850)
0810 18:57:38.689353 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.689368 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.689459 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.689889 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.689892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.689919 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:38.690009 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.690036 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.691703 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.691707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.692717 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.692727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.693832 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.693836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.693931 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.693963 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.694009 (+ 46us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.694081 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"mutex_wait_us":32,"thread_start_us":78,"threads_started":1}
I20260810 18:57:38.695672 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.688037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:32859 (local address 127.0.0.1:34866)
0810 18:57:38.688692 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.688744 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.688883 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.689758 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.689763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.689787 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:38.689960 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.689968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.691607 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.691611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.693863 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.693871 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.695225 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.695229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.695338 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.695383 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.695451 (+ 68us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.695547 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":4,"thread_start_us":123,"threads_started":1}
I20260810 18:57:38.696072 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.688182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.30.153.130:32859)
0810 18:57:38.689241 (+ 1059us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.689253 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.689278 (+ 25us) server_negotiation.cc:407] Connection header received
0810 18:57:38.689363 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.689368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.689575 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.689732 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:38.690588 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.691448 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.693992 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.694894 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.695020 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.695126 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.695845 (+ 719us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.695942 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":943,"thread_start_us":124,"threads_started":1}
I20260810 18:57:38.697769 28327 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef4d5cdafab4f88826efaee2328a4db (DEFAULT_TABLE table=test-table [id=6415bf723eab4bc888a2e69ca1fe2723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:38.698163 28758 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef4d5cdafab4f88826efaee2328a4db (DEFAULT_TABLE table=test-table [id=6415bf723eab4bc888a2e69ca1fe2723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:38.698969 28327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef4d5cdafab4f88826efaee2328a4db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:38.698727 28549 tablet_service.cc:1467] Processing CreateTablet for tablet 3ef4d5cdafab4f88826efaee2328a4db (DEFAULT_TABLE table=test-table [id=6415bf723eab4bc888a2e69ca1fe2723]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:38.699285 28758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef4d5cdafab4f88826efaee2328a4db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:38.699707 28549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ef4d5cdafab4f88826efaee2328a4db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:38.705229 28899 tablet_bootstrap.cc:492] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f: Bootstrap starting.
I20260810 18:57:38.706782 28900 tablet_bootstrap.cc:492] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: Bootstrap starting.
I20260810 18:57:38.707823 28901 tablet_bootstrap.cc:492] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: Bootstrap starting.
I20260810 18:57:38.707896 28899 tablet_bootstrap.cc:654] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:38.708818 28899 log.cc:824] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:38.708988 28900 tablet_bootstrap.cc:654] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:38.709800 28901 tablet_bootstrap.cc:654] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:38.710248 28900 log.cc:824] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:38.711104 28899 tablet_bootstrap.cc:492] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f: No bootstrap required, opened a new log
I20260810 18:57:38.711131 28901 log.cc:824] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:38.711323 28899 ts_tablet_manager.cc:1397] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:57:38.712313 28900 tablet_bootstrap.cc:492] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: No bootstrap required, opened a new log
I20260810 18:57:38.712538 28900 ts_tablet_manager.cc:1397] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:57:38.715833 28901 tablet_bootstrap.cc:492] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: No bootstrap required, opened a new log
I20260810 18:57:38.715822 28899 raft_consensus.cc:348] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.716087 28901 ts_tablet_manager.cc:1397] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260810 18:57:38.716140 28899 raft_consensus.cc:374] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:38.716214 28899 raft_consensus.cc:729] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f175a5f15f994f0aa1ee0cedcf9d319f, State: Initialized, Role: FOLLOWER
I20260810 18:57:38.716476 28900 raft_consensus.cc:348] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.716672 28899 consensus_queue.cc:260] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.716756 28900 raft_consensus.cc:374] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:38.716810 28900 raft_consensus.cc:729] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87a3230760e045c4a8d36b8e2f216e1d, State: Initialized, Role: FOLLOWER
I20260810 18:57:38.717260 28900 consensus_queue.cc:260] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.717407 28439 heartbeater.cc:503] Master 127.30.153.190:42511 was elected leader, sending a full tablet report...
I20260810 18:57:38.717777 28899 ts_tablet_manager.cc:1428] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:57:38.718225 28870 heartbeater.cc:503] Master 127.30.153.190:42511 was elected leader, sending a full tablet report...
I20260810 18:57:38.719147 28900 ts_tablet_manager.cc:1428] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260810 18:57:38.719914 28901 raft_consensus.cc:348] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.720284 28901 raft_consensus.cc:374] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:38.720352 28901 raft_consensus.cc:729] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff2ace79c463416c917c7917425f4bab, State: Initialized, Role: FOLLOWER
I20260810 18:57:38.720788 28901 consensus_queue.cc:260] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.721493 28655 heartbeater.cc:503] Master 127.30.153.190:42511 was elected leader, sending a full tablet report...
I20260810 18:57:38.721791 28901 ts_tablet_manager.cc:1428] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260810 18:57:38.747553 28907 raft_consensus.cc:482] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:38.747742 28907 raft_consensus.cc:504] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.748917 28907 leader_election.cc:288] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff2ace79c463416c917c7917425f4bab (127.30.153.130:32859), f175a5f15f994f0aa1ee0cedcf9d319f (127.30.153.129:42377)
I20260810 18:57:38.752738 28906 raft_consensus.cc:482] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:38.752868 28872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.749170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:32859 (local address 127.30.153.131:38807)
0810 18:57:38.749289 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.749308 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.749408 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.749787 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.749790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.749811 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:38.749938 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.749943 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.750747 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.750752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.751631 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.751641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.752427 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.752431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.752545 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.752571 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.752650 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.752724 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260810 18:57:38.752950 28906 raft_consensus.cc:504] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.753161 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.749452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:38807 (local address 127.30.153.130:32859)
0810 18:57:38.749572 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.749575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.749587 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:57:38.749632 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.749634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.749675 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.749771 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:38.750069 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.750634 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.751758 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.752301 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.752428 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.752480 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.752966 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.753045 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 18:57:38.753412 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.749285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:36335 (local address 127.30.153.129:42377)
0810 18:57:38.749433 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.749438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.749692 (+ 254us) server_negotiation.cc:407] Connection header received
0810 18:57:38.749842 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.749845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.749938 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.750028 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:38.750331 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.750847 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.751903 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.752470 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.752584 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.752656 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.753215 (+ 559us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.753300 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 18:57:38.753736 28910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.749171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:42377 (local address 127.30.153.131:36335)
0810 18:57:38.749590 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.749601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.749729 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.750040 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.750042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.750058 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:38.750190 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.750196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.750967 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.750970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.751756 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.751765 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.752713 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.752716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.752801 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.752821 (+ 20us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.753191 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.753395 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":52,"thread_start_us":149,"threads_started":1}
I20260810 18:57:38.753759 28583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef4d5cdafab4f88826efaee2328a4db" candidate_uuid: "87a3230760e045c4a8d36b8e2f216e1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2ace79c463416c917c7917425f4bab" is_pre_election: true
I20260810 18:57:38.754034 28583 raft_consensus.cc:2454] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a3230760e045c4a8d36b8e2f216e1d in term 0.
I20260810 18:57:38.754364 28361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef4d5cdafab4f88826efaee2328a4db" candidate_uuid: "87a3230760e045c4a8d36b8e2f216e1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" is_pre_election: true
I20260810 18:57:38.754592 28361 raft_consensus.cc:2454] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87a3230760e045c4a8d36b8e2f216e1d in term 0.
I20260810 18:57:38.754592 28692 leader_election.cc:302] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87a3230760e045c4a8d36b8e2f216e1d, ff2ace79c463416c917c7917425f4bab; no voters:
I20260810 18:57:38.755492 28906 leader_election.cc:288] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff2ace79c463416c917c7917425f4bab (127.30.153.130:32859), 87a3230760e045c4a8d36b8e2f216e1d (127.30.153.131:41261)
I20260810 18:57:38.755446 28907 raft_consensus.cc:2781] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:57:38.755568 28907 raft_consensus.cc:482] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:57:38.755638 28907 raft_consensus.cc:3037] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:38.756888 28907 raft_consensus.cc:504] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.757553 28361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef4d5cdafab4f88826efaee2328a4db" candidate_uuid: "87a3230760e045c4a8d36b8e2f216e1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f"
I20260810 18:57:38.757575 28583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef4d5cdafab4f88826efaee2328a4db" candidate_uuid: "87a3230760e045c4a8d36b8e2f216e1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2ace79c463416c917c7917425f4bab"
I20260810 18:57:38.757681 28361 raft_consensus.cc:3037] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:38.757712 28583 raft_consensus.cc:3037] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:38.759001 28583 raft_consensus.cc:2454] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a3230760e045c4a8d36b8e2f216e1d in term 1.
I20260810 18:57:38.759021 28361 raft_consensus.cc:2454] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87a3230760e045c4a8d36b8e2f216e1d in term 1.
I20260810 18:57:38.759043 28893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.755007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:45135 (local address 127.30.153.130:32859)
0810 18:57:38.755120 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.755123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.755134 (+ 11us) server_negotiation.cc:407] Connection header received
0810 18:57:38.755178 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.755179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.755238 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.755311 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:38.755693 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.756340 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.757854 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.758453 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.758562 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.758608 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.758810 (+ 202us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.758896 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:57:38.759248 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.754611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:32859 (local address 127.30.153.129:45135)
0810 18:57:38.754900 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.754931 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.755031 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.755412 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.755417 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.755437 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:38.755552 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.755558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.756471 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.756474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.757696 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.757708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.758554 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.758557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.758668 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.758693 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.758779 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.758867 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":130,"threads_started":1}
I20260810 18:57:38.759315 28692 leader_election.cc:302] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87a3230760e045c4a8d36b8e2f216e1d, ff2ace79c463416c917c7917425f4bab; no voters:
I20260810 18:57:38.759451 28907 leader_election.cc:288] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [CANDIDATE]: Term 1 election: Requested vote from peers ff2ace79c463416c917c7917425f4bab (127.30.153.130:32859), f175a5f15f994f0aa1ee0cedcf9d319f (127.30.153.129:42377)
I20260810 18:57:38.759582 28907 raft_consensus.cc:2781] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:38.759675 28907 raft_consensus.cc:686] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 1 LEADER]: Becoming Leader. State: Replica: 87a3230760e045c4a8d36b8e2f216e1d, State: Running, Role: LEADER
I20260810 18:57:38.759821 28583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef4d5cdafab4f88826efaee2328a4db" candidate_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff2ace79c463416c917c7917425f4bab" is_pre_election: true
I20260810 18:57:38.759987 28583 raft_consensus.cc:2379] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f175a5f15f994f0aa1ee0cedcf9d319f in current term 1: Already voted for candidate 87a3230760e045c4a8d36b8e2f216e1d in this term.
I20260810 18:57:38.760020 28907 consensus_queue.cc:237] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } }
I20260810 18:57:38.761287 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.755328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:58771 (local address 127.30.153.131:41261)
0810 18:57:38.755428 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.755432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.755775 (+ 343us) server_negotiation.cc:407] Connection header received
0810 18:57:38.755880 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.755883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.755920 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.756005 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:38.756269 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.756957 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.760231 (+ 3274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.760763 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.760887 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.760930 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.761126 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.761190 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 18:57:38.761309 28913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.755252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:41261 (local address 127.30.153.129:58771)
0810 18:57:38.755687 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.755703 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:38.755802 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:38.756027 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:38.756029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:38.756045 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:38.756160 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.756165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.757092 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.757095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.760075 (+ 2980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:38.760086 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.760885 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.760888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:38.760992 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.761019 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 18:57:38.761104 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 18:57:38.761169 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":113,"threads_started":1}
I20260810 18:57:38.761894 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ef4d5cdafab4f88826efaee2328a4db" candidate_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87a3230760e045c4a8d36b8e2f216e1d" is_pre_election: true
I20260810 18:57:38.762545 28252 leader_election.cc:302] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f175a5f15f994f0aa1ee0cedcf9d319f; no voters: 87a3230760e045c4a8d36b8e2f216e1d, ff2ace79c463416c917c7917425f4bab
I20260810 18:57:38.762993 28906 raft_consensus.cc:2726] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 18:57:38.763345 28137 catalog_manager.cc:5495] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 87a3230760e045c4a8d36b8e2f216e1d (127.30.153.131). New cstate: current_term: 1 leader_uuid: "87a3230760e045c4a8d36b8e2f216e1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87a3230760e045c4a8d36b8e2f216e1d" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 41261 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 } health_report { overall_health: UNKNOWN } } }
I20260810 18:57:38.776746 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.772965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.30.153.131:41261)
0810 18:57:38.773109 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:38.773114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:38.773127 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:57:38.773274 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:38.773277 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:38.773339 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:38.773445 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:57:38.773736 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.774233 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.775284 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:38.775510 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:38.775617 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:38.775957 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:57:38.776353 (+ 396us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:57:38.776486 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:38.776553 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 18:57:38.776604 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:57:38.782609 28583 raft_consensus.cc:1261] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 1 FOLLOWER]: Refusing update from remote peer 87a3230760e045c4a8d36b8e2f216e1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:38.782635 28361 raft_consensus.cc:1261] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 1 FOLLOWER]: Refusing update from remote peer 87a3230760e045c4a8d36b8e2f216e1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:38.783304 28907 consensus_queue.cc:1035] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f175a5f15f994f0aa1ee0cedcf9d319f" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 42377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:38.783505 28914 consensus_queue.cc:1035] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff2ace79c463416c917c7917425f4bab" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 32859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:38.790624 28917 mvcc.cc:205] Tried to move back new op lower bound from 7317046307966377984 to 7317046307884126208. Current Snapshot: MvccSnapshot[applied={T|T < 7317046307966377984}]
I20260810 18:57:38.791468 28919 mvcc.cc:205] Tried to move back new op lower bound from 7317046307966377984 to 7317046307884126208. Current Snapshot: MvccSnapshot[applied={T|T < 7317046307966377984}]
I20260810 18:57:38.800330 28133 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:38.800493 28133 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:38.802944 28133 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fe88820bf3f44bba0e13e23518da547: Sending DeleteTablet for 3 replicas of tablet 3ef4d5cdafab4f88826efaee2328a4db
I20260810 18:57:38.803741 28549 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef4d5cdafab4f88826efaee2328a4db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:38 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:34866
I20260810 18:57:38.803781 28327 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef4d5cdafab4f88826efaee2328a4db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:38 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:39440
I20260810 18:57:38.804266 28758 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ef4d5cdafab4f88826efaee2328a4db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:38 UTC) from {username='kudu', principal='kudu/127.30.153.190@KRBTEST.COM'} at 127.0.0.1:35850
I20260810 18:57:38.804301 28933 tablet_replica.cc:330] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f: stopping tablet replica
I20260810 18:57:38.804663 28933 raft_consensus.cc:2229] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:38.804803 28940 tablet_replica.cc:330] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: stopping tablet replica
I20260810 18:57:38.804999 28933 raft_consensus.cc:2258] T 3ef4d5cdafab4f88826efaee2328a4db P f175a5f15f994f0aa1ee0cedcf9d319f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:38.805249 28940 raft_consensus.cc:2229] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:57:38.805392 28939 tablet_replica.cc:330] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: stopping tablet replica
I20260810 18:57:38.805697 28939 raft_consensus.cc:2229] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:38.805703 28940 raft_consensus.cc:2258] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:38.805858 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 28228
I20260810 18:57:38.805984 28939 raft_consensus.cc:2258] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:38.812916 28939 ts_tablet_manager.cc:1905] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:38.815768 28939 ts_tablet_manager.cc:1918] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:38.815879 28939 log.cc:1196] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/3ef4d5cdafab4f88826efaee2328a4db
I20260810 18:57:38.816224 28939 ts_tablet_manager.cc:1939] T 3ef4d5cdafab4f88826efaee2328a4db P ff2ace79c463416c917c7917425f4bab: Deleting consensus metadata
I20260810 18:57:38.817603 28101 catalog_manager.cc:4841] TS ff2ace79c463416c917c7917425f4bab (127.30.153.130:32859): tablet 3ef4d5cdafab4f88826efaee2328a4db (table test-table [id=6415bf723eab4bc888a2e69ca1fe2723]) successfully deleted
I20260810 18:57:38.817646 28940 ts_tablet_manager.cc:1905] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:38.821346 28940 ts_tablet_manager.cc:1918] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:38.821465 28940 log.cc:1196] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/3ef4d5cdafab4f88826efaee2328a4db
I20260810 18:57:38.821830 28940 ts_tablet_manager.cc:1939] T 3ef4d5cdafab4f88826efaee2328a4db P 87a3230760e045c4a8d36b8e2f216e1d: Deleting consensus metadata
I20260810 18:57:38.823153 28100 catalog_manager.cc:4841] TS 87a3230760e045c4a8d36b8e2f216e1d (127.30.153.131:41261): tablet 3ef4d5cdafab4f88826efaee2328a4db (table test-table [id=6415bf723eab4bc888a2e69ca1fe2723]) successfully deleted
W20260810 18:57:38.823788 28106 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:42377 (error 108)
I20260810 18:57:38.823978 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 28446
I20260810 18:57:38.831440 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:38.831035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:42377 (local address 127.0.0.1:39456)
0810 18:57:38.831219 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:38.831312 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:42377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 18:57:38.831643 28106 catalog_manager.cc:4584] TS f175a5f15f994f0aa1ee0cedcf9d319f (127.30.153.129:42377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3ef4d5cdafab4f88826efaee2328a4db: Network error: Client connection negotiation failed: client connection to 127.30.153.129:42377: connect: Connection refused (error 111)
I20260810 18:57:38.835328 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 28663
I20260810 18:57:38.852927 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 28072
2026-08-10T18:57:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2630 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T18:57:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:38Z Disabled control of system clock
I20260810 18:57:38.889721 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:43463
--webserver_interface=127.30.153.190
--webserver_port=0
--builtin_ntp_servers=127.30.153.148:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:43463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:57:39.004170 28949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:39.004439 28949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:39.004524 28949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:39.011739 28949 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:39.011821 28949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:39.011847 28949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:39.011889 28949 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:39.011921 28949 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:39.017599 28949 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:43463
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:43463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:39.018841 28949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:39.020270 28949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:39.028993 28962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:39.028980 28960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:39.028955 28959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:39.029358 28949 server_base.cc:1034] running on GCE node
I20260810 18:57:39.029892 28949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:39.031042 28949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:39.032245 28949 hybrid_clock.cc:648] HybridClock initialized: now 1786388259032224 us; error 32 us; skew 500 ppm
I20260810 18:57:39.034044 28949 webserver.cc:460] Webserver started at http://127.30.153.190:42975/ using document root <none> and password file <none>
I20260810 18:57:39.034698 28949 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:39.034821 28949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:39.035059 28949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:39.037134 28949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "ce37f23a26604d0da55eeb919fd3a11d"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "f8ece6e06b43d9756146f9665ff472bc"
server_key_iv: "4d5ba8eef50104366fc9f4ee70548a51"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.037719 28949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "ce37f23a26604d0da55eeb919fd3a11d"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "f8ece6e06b43d9756146f9665ff472bc"
server_key_iv: "4d5ba8eef50104366fc9f4ee70548a51"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.041455 28949 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 18:57:39.044014 28973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.045306 28949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:57:39.045470 28949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "ce37f23a26604d0da55eeb919fd3a11d"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "f8ece6e06b43d9756146f9665ff472bc"
server_key_iv: "4d5ba8eef50104366fc9f4ee70548a51"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.045594 28949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:39.055476 28949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:39.056116 28949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:39.056319 28949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:39.063319 28949 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:43463
I20260810 18:57:39.063369 29055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:43463 every 8 connection(s)
I20260810 18:57:39.064280 28949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:39.066093 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 28949
I20260810 18:57:39.066241 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:39.066618 31334 external_mini_cluster.cc:1428] Setting key d2c6ccca4169f35f4b6cd34c75de5896
I20260810 18:57:39.067883 29056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:39.073469 29056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d: Bootstrap starting.
I20260810 18:57:39.075992 29056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:39.076138 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.068176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47306 (local address 127.30.153.190:43463)
0810 18:57:39.068842 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.068857 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.068893 (+ 36us) server_negotiation.cc:407] Connection header received
0810 18:57:39.069537 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.069552 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.069935 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.070324 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:39.070808 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.071785 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.072710 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.073196 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.073376 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.074313 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:39.074318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:39.074331 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:39.074775 (+ 444us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:39.074831 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:39.074853 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:39.074970 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.075134 (+ 164us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.075422 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":163,"threads_started":1}
I20260810 18:57:39.076956 29056 log.cc:824] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:39.079070 29056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d: No bootstrap required, opened a new log
I20260810 18:57:39.082711 29056 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce37f23a26604d0da55eeb919fd3a11d" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 43463 } }
I20260810 18:57:39.082934 29056 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:39.082970 29056 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce37f23a26604d0da55eeb919fd3a11d, State: Initialized, Role: FOLLOWER
I20260810 18:57:39.083377 29056 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce37f23a26604d0da55eeb919fd3a11d" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 43463 } }
I20260810 18:57:39.083500 29056 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:39.083547 29056 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:39.083647 29056 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:39.084591 29056 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce37f23a26604d0da55eeb919fd3a11d" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 43463 } }
I20260810 18:57:39.084967 29056 leader_election.cc:302] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce37f23a26604d0da55eeb919fd3a11d; no voters:
I20260810 18:57:39.085379 29056 leader_election.cc:288] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:39.085552 29064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:39.085803 29064 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [term 1 LEADER]: Becoming Leader. State: Replica: ce37f23a26604d0da55eeb919fd3a11d, State: Running, Role: LEADER
I20260810 18:57:39.086159 29064 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce37f23a26604d0da55eeb919fd3a11d" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 43463 } }
I20260810 18:57:39.086711 29056 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:39.087913 29065 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce37f23a26604d0da55eeb919fd3a11d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce37f23a26604d0da55eeb919fd3a11d" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 43463 } } }
I20260810 18:57:39.088063 29065 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:39.088485 29066 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce37f23a26604d0da55eeb919fd3a11d. Latest consensus state: current_term: 1 leader_uuid: "ce37f23a26604d0da55eeb919fd3a11d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce37f23a26604d0da55eeb919fd3a11d" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 43463 } } }
I20260810 18:57:39.088617 29066 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:39.088555 29078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:39.091784 29078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:39.098114 29078 catalog_manager.cc:1347] Generated new cluster ID: f232084d1b144f408a5d3880adb6f3db
I20260810 18:57:39.098192 29078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:39.106832 29078 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:39.107738 29078 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:39.116140 29078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d: Generated new TSK 0
I20260810 18:57:39.116897 29078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:39.131445 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.129:0
--local_ip_for_outbound_sockets=127.30.153.129
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:43463
--builtin_ntp_servers=127.30.153.148:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 18:57:39.247262 29088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:39.247520 29088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:39.247556 29088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:39.252332 29088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:39.252393 29088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:39.252449 29088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.129
I20260810 18:57:39.257222 29088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.153.129
--webserver_port=0
--tserver_master_addrs=127.30.153.190:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.129
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:39.258363 29088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:39.259845 29088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:57:39.270097 29088 server_base.cc:1034] running on GCE node
W20260810 18:57:39.270056 29096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:39.270056 29099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:39.270077 29094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:39.270759 29088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:39.271461 29088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:39.272668 29088 hybrid_clock.cc:648] HybridClock initialized: now 1786388259272637 us; error 44 us; skew 500 ppm
I20260810 18:57:39.274573 29088 webserver.cc:460] Webserver started at http://127.30.153.129:39789/ using document root <none> and password file <none>
I20260810 18:57:39.275193 29088 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:39.275292 29088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:39.275537 29088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:39.277616 29088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/data/instance:
uuid: "f6aad75ee27140b6bb9c6838878682b7"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "369ba426babe68970e17cadc6ccf3dea"
server_key_iv: "3714c0fc156d4bdb1e905b7beddfabeb"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.278203 29088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance:
uuid: "f6aad75ee27140b6bb9c6838878682b7"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "369ba426babe68970e17cadc6ccf3dea"
server_key_iv: "3714c0fc156d4bdb1e905b7beddfabeb"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.281984 29088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260810 18:57:39.284591 29108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.286278 29088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:57:39.286401 29088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/wal
uuid: "f6aad75ee27140b6bb9c6838878682b7"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "369ba426babe68970e17cadc6ccf3dea"
server_key_iv: "3714c0fc156d4bdb1e905b7beddfabeb"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.286530 29088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:39.303391 29088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:39.304136 29088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:39.304358 29088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:39.304986 29088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:39.306038 29088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:39.306088 29088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.306167 29088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:39.306200 29088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.315809 29088 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.129:45065
I20260810 18:57:39.315840 29293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.129:45065 every 8 connection(s)
I20260810 18:57:39.316730 29088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/data/info.pb
I20260810 18:57:39.319074 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 29088
I20260810 18:57:39.319242 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-0/wal/instance
I20260810 18:57:39.319638 31334 external_mini_cluster.cc:1428] Setting key 1cb18e0c909442bd243de0f646e517c0
I20260810 18:57:39.321944 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.130:0
--local_ip_for_outbound_sockets=127.30.153.130
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:43463
--builtin_ntp_servers=127.30.153.148:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:57:39.327400 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.318614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.129:37807 (local address 127.30.153.190:43463)
0810 18:57:39.318771 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.318775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.320078 (+ 1303us) server_negotiation.cc:407] Connection header received
0810 18:57:39.321900 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.321905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.321972 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.322086 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:39.323408 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.323954 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.325283 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.325533 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.325653 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.326837 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:39.326841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:39.326844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:39.326865 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:39.326887 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:39.326894 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:39.326981 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.327167 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.327248 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:57:39.328274 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.319009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:43463 (local address 127.30.153.129:37807)
0810 18:57:39.319878 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:39.319936 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:39.321645 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:39.322314 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:39.322326 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:39.322469 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:39.323220 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.323244 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.324207 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.324234 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.325134 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.325145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.325701 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.325705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.325878 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.326562 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:39.326564 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:39.326605 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:39.326671 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:39.326985 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:39.326997 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:39.327006 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:57:39.327142 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 18:57:39.327444 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":163,"threads_started":1}
I20260810 18:57:39.329871 29294 heartbeater.cc:347] Connected to a master server at 127.30.153.190:43463
I20260810 18:57:39.330140 29294 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:39.330762 29294 heartbeater.cc:511] Master 127.30.153.190:43463 requested a full tablet report, sending...
I20260810 18:57:39.332603 28995 ts_manager.cc:194] Registered new tserver with Master: f6aad75ee27140b6bb9c6838878682b7 (127.30.153.129:45065)
I20260810 18:57:39.334450 28995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.129:37807
W20260810 18:57:39.437652 29298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:39.437888 29298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:39.437925 29298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:39.441722 29298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:39.441764 29298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:39.441813 29298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.130
I20260810 18:57:39.446280 29298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.153.130
--webserver_port=0
--tserver_master_addrs=127.30.153.190:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.130
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:39.447268 29298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:39.448597 29298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:39.458304 29307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:39.458251 29313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:39.458245 29309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:39.458586 29298 server_base.cc:1034] running on GCE node
I20260810 18:57:39.459002 29298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:39.459676 29298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:39.460893 29298 hybrid_clock.cc:648] HybridClock initialized: now 1786388259460880 us; error 28 us; skew 500 ppm
I20260810 18:57:39.462966 29298 webserver.cc:460] Webserver started at http://127.30.153.130:43009/ using document root <none> and password file <none>
I20260810 18:57:39.463570 29298 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:39.463667 29298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:39.463932 29298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:39.466022 29298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/data/instance:
uuid: "bae84caa79704668bf91d3f9d769eac9"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "9b91870976f98828ef106b09c07a67f7"
server_key_iv: "3d3d326ac9bc705e72c8d85b0ff76d43"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.466672 29298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance:
uuid: "bae84caa79704668bf91d3f9d769eac9"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "9b91870976f98828ef106b09c07a67f7"
server_key_iv: "3d3d326ac9bc705e72c8d85b0ff76d43"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.470583 29298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:39.473089 29322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.474334 29298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:39.474512 29298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/wal
uuid: "bae84caa79704668bf91d3f9d769eac9"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "9b91870976f98828ef106b09c07a67f7"
server_key_iv: "3d3d326ac9bc705e72c8d85b0ff76d43"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.474633 29298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:39.503006 29298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:39.503729 29298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:39.503940 29298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:39.504578 29298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:39.505564 29298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:39.505640 29298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.505718 29298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:39.505771 29298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.514395 29298 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.130:41413
I20260810 18:57:39.514503 29499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.130:41413 every 8 connection(s)
I20260810 18:57:39.515357 29298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/data/info.pb
I20260810 18:57:39.519618 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 29298
I20260810 18:57:39.519739 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/wal/instance
I20260810 18:57:39.520123 31334 external_mini_cluster.cc:1428] Setting key b1bbad235cd3a202c53a4123ea504ddd
I20260810 18:57:39.522516 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.153.131:0
--local_ip_for_outbound_sockets=127.30.153.131
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:43463
--builtin_ntp_servers=127.30.153.148:39935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 18:57:39.525641 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.517248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:55549 (local address 127.30.153.190:43463)
0810 18:57:39.517409 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.517414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.518573 (+ 1159us) server_negotiation.cc:407] Connection header received
0810 18:57:39.520036 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.520039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.520101 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.520236 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:39.521266 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.521908 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.523369 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.523760 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.523890 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.525021 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:39.525028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:39.525046 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:39.525064 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:39.525094 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:39.525107 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:39.525195 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.525380 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.525462 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 18:57:39.526535 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.517684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:43463 (local address 127.30.153.130:55549)
0810 18:57:39.518370 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:39.518459 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:39.519803 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:39.520449 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:39.520456 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:39.520560 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:39.521121 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.521135 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.522055 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.522061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.523239 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.523250 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.523918 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.523921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.524064 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.524748 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:39.524750 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:39.524790 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:39.524853 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:39.525231 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:39.525239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:39.525243 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 18:57:39.525345 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:57:39.525627 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260810 18:57:39.527734 29500 heartbeater.cc:347] Connected to a master server at 127.30.153.190:43463
I20260810 18:57:39.527984 29500 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:39.528512 29500 heartbeater.cc:511] Master 127.30.153.190:43463 requested a full tablet report, sending...
I20260810 18:57:39.529637 28995 ts_manager.cc:194] Registered new tserver with Master: bae84caa79704668bf91d3f9d769eac9 (127.30.153.130:41413)
I20260810 18:57:39.530267 28995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.130:55549
W20260810 18:57:39.638324 29509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:39.638701 29509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:39.638783 29509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:39.642791 29509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:39.642879 29509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:39.642974 29509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.153.131
I20260810 18:57:39.647791 29509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:39935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.153.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.153.131
--webserver_port=0
--tserver_master_addrs=127.30.153.190:43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.153.131
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:39.648948 29509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:39.650471 29509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:39.661224 29520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:39.661253 29509 server_base.cc:1034] running on GCE node
W20260810 18:57:39.661206 29516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:39.661206 29518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:39.662097 29509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:39.662858 29509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:39.664036 29509 hybrid_clock.cc:648] HybridClock initialized: now 1786388259664003 us; error 44 us; skew 500 ppm
I20260810 18:57:39.665985 29509 webserver.cc:460] Webserver started at http://127.30.153.131:45717/ using document root <none> and password file <none>
I20260810 18:57:39.666774 29509 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:39.666846 29509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:39.667096 29509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:39.669293 29509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/data/instance:
uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "e5ab776fad33be92cc57241b4188a434"
server_key_iv: "46e2b217c3f8f9fb4feea7b18dd1d4ba"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.669941 29509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance:
uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "e5ab776fad33be92cc57241b4188a434"
server_key_iv: "46e2b217c3f8f9fb4feea7b18dd1d4ba"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.673856 29509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260810 18:57:39.676604 29537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.677996 29509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:57:39.678179 29509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/wal
uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386"
format_stamp: "Formatted at 2026-08-10 18:57:39 on dist-test-slave-68xk"
server_key: "e5ab776fad33be92cc57241b4188a434"
server_key_iv: "46e2b217c3f8f9fb4feea7b18dd1d4ba"
server_key_version: "encryptionkey@0"
I20260810 18:57:39.678288 29509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:39.697016 29509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:39.697907 29509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:39.698137 29509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:39.698926 29509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 18:57:39.699981 29509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 18:57:39.700034 29509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.700078 29509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 18:57:39.700095 29509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:39.709743 29509 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.131:35289
I20260810 18:57:39.709767 29732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.131:35289 every 8 connection(s)
I20260810 18:57:39.710908 29509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/data/info.pb
I20260810 18:57:39.720722 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.712901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.131:58023 (local address 127.30.153.190:43463)
0810 18:57:39.713042 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.713045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.714137 (+ 1092us) server_negotiation.cc:407] Connection header received
0810 18:57:39.715693 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.715697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.715763 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.715875 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:39.716829 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.717371 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.718580 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.718814 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.718935 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.720150 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:39.720153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:39.720155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:39.720169 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:39.720191 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:39.720199 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:39.720292 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.720477 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.720544 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:57:39.720747 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 29509
I20260810 18:57:39.721005 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/wal/instance
I20260810 18:57:39.721370 31334 external_mini_cluster.cc:1428] Setting key cf815d45871994b8e67d0e316ba28e1e
I20260810 18:57:39.721783 29735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.713263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:43463 (local address 127.30.153.131:58023)
0810 18:57:39.713974 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:39.714016 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:39.715459 (+ 1443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:39.716070 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:39.716077 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:39.716164 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:57:39.716661 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.716673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.717513 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.717519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.718443 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.718453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.718984 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.718989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.719162 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.719847 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 18:57:39.719848 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:57:39.719891 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 18:57:39.719957 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:57:39.720294 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:57:39.720306 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:57:39.720315 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 18:57:39.720460 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 18:57:39.720748 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":135,"threads_started":1}
I20260810 18:57:39.723209 29733 heartbeater.cc:347] Connected to a master server at 127.30.153.190:43463
I20260810 18:57:39.723483 29733 heartbeater.cc:464] Registering TS with master...
I20260810 18:57:39.724160 29733 heartbeater.cc:511] Master 127.30.153.190:43463 requested a full tablet report, sending...
I20260810 18:57:39.725257 28995 ts_manager.cc:194] Registered new tserver with Master: 8dee758b94dd4b3e84f6a5fe4a7dc386 (127.30.153.131:35289)
I20260810 18:57:39.725826 28995 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.153.131:58023
I20260810 18:57:39.736163 31334 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 18:57:39.742758 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.738396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47312 (local address 127.30.153.190:43463)
0810 18:57:39.738595 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.738601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.738780 (+ 179us) server_negotiation.cc:407] Connection header received
0810 18:57:39.738951 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.738956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.739012 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.739118 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:39.739433 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.739952 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.740911 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.741150 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.741269 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.742173 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:39.742176 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:39.742179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:39.742191 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:39.742212 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:39.742226 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:39.742338 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.742532 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.742595 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 18:57:39.745854 28995 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 18:57:39.749043 28995 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 18:57:39.764279 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.758217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35289 (local address 127.0.0.1:41522)
0810 18:57:39.758528 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:39.758545 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:39.758638 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:39.759889 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:39.759895 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:39.759918 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:39.760062 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.760069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.761285 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.761289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.762633 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.762644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.763714 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.763718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.763835 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.763875 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 18:57:39.763991 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 18:57:39.764109 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
I20260810 18:57:39.764616 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.758772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.30.153.131:35289)
0810 18:57:39.759382 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.759395 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.759421 (+ 26us) server_negotiation.cc:407] Connection header received
0810 18:57:39.759499 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.759504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.759697 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.759888 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:39.760198 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.761147 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.762765 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.763572 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.763720 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.763908 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.764347 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.764441 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":104,"threads_started":1}
I20260810 18:57:39.764916 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.758904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.30.153.130:41413)
0810 18:57:39.759465 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.759477 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.759496 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:57:39.759560 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.759564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.759799 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.759930 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:39.760415 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.761558 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.763022 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.763588 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.763739 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.763909 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.764626 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.764752 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":144,"threads_started":1}
I20260810 18:57:39.764336 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.758688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.130:41413 (local address 127.0.0.1:50264)
0810 18:57:39.758974 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:39.758988 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:39.759077 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:39.760095 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:39.760097 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:39.760113 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:39.760286 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.760294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.761723 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.761726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.762888 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.762899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.763977 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.763984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.764076 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.764097 (+ 21us) client_negotiation.cc:769] Sending connection context
0810 18:57:39.764163 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 18:57:39.764226 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":15,"thread_start_us":128,"threads_started":1}
I20260810 18:57:39.765367 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.757823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45065 (local address 127.0.0.1:41516)
0810 18:57:39.758235 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:39.758295 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:39.758451 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:39.760077 (+ 1626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:39.760080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:39.760099 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:39.760218 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.760224 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.761692 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.761695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.762938 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.762946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.764004 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.764007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.764098 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.764120 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 18:57:39.764187 (+ 67us) client_negotiation.cc:240] Negotiation successful
0810 18:57:39.764269 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":159,"thread_start_us":120,"threads_started":1}
I20260810 18:57:39.767436 29623 tablet_service.cc:1467] Processing CreateTablet for tablet 0359679582eb49749cde5a84a87fbf87 (DEFAULT_TABLE table=test-table [id=5fbb5b46567b401ca8590e95439afaea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:39.767580 29392 tablet_service.cc:1467] Processing CreateTablet for tablet 0359679582eb49749cde5a84a87fbf87 (DEFAULT_TABLE table=test-table [id=5fbb5b46567b401ca8590e95439afaea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:39.767954 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.758041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41516 (local address 127.30.153.129:45065)
0810 18:57:39.759348 (+ 1307us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.759360 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.759388 (+ 28us) server_negotiation.cc:407] Connection header received
0810 18:57:39.759469 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.759475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.759704 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.759904 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:39.760344 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.761558 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.763078 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.763848 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.764014 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.764182 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.764236 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.764337 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1177,"thread_start_us":122,"threads_started":1}
I20260810 18:57:39.768522 29392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0359679582eb49749cde5a84a87fbf87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:39.768545 29623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0359679582eb49749cde5a84a87fbf87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:39.770763 29192 tablet_service.cc:1467] Processing CreateTablet for tablet 0359679582eb49749cde5a84a87fbf87 (DEFAULT_TABLE table=test-table [id=5fbb5b46567b401ca8590e95439afaea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 18:57:39.771832 29192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0359679582eb49749cde5a84a87fbf87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:39.775266 29758 tablet_bootstrap.cc:492] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Bootstrap starting.
I20260810 18:57:39.777436 29758 tablet_bootstrap.cc:654] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:39.777951 29760 tablet_bootstrap.cc:492] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Bootstrap starting.
I20260810 18:57:39.778462 29758 log.cc:824] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:39.779220 29764 tablet_bootstrap.cc:492] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7: Bootstrap starting.
I20260810 18:57:39.780346 29760 tablet_bootstrap.cc:654] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:39.780812 29758 tablet_bootstrap.cc:492] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: No bootstrap required, opened a new log
I20260810 18:57:39.781013 29758 ts_tablet_manager.cc:1397] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 18:57:39.781191 29760 log.cc:824] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:39.781927 29764 tablet_bootstrap.cc:654] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:39.782718 29764 log.cc:824] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:39.783535 29760 tablet_bootstrap.cc:492] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: No bootstrap required, opened a new log
I20260810 18:57:39.783728 29760 ts_tablet_manager.cc:1397] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 18:57:39.784607 29764 tablet_bootstrap.cc:492] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7: No bootstrap required, opened a new log
I20260810 18:57:39.784816 29764 ts_tablet_manager.cc:1397] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260810 18:57:39.785449 29758 raft_consensus.cc:348] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.785765 29758 raft_consensus.cc:374] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:39.785828 29758 raft_consensus.cc:729] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dee758b94dd4b3e84f6a5fe4a7dc386, State: Initialized, Role: FOLLOWER
I20260810 18:57:39.786331 29758 consensus_queue.cc:260] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.787376 29758 ts_tablet_manager.cc:1428] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260810 18:57:39.787770 29733 heartbeater.cc:503] Master 127.30.153.190:43463 was elected leader, sending a full tablet report...
I20260810 18:57:39.788262 29760 raft_consensus.cc:348] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.788451 29760 raft_consensus.cc:374] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:39.788483 29760 raft_consensus.cc:729] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae84caa79704668bf91d3f9d769eac9, State: Initialized, Role: FOLLOWER
I20260810 18:57:39.788972 29760 consensus_queue.cc:260] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.789408 29764 raft_consensus.cc:348] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.789616 29500 heartbeater.cc:503] Master 127.30.153.190:43463 was elected leader, sending a full tablet report...
I20260810 18:57:39.789657 29764 raft_consensus.cc:374] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:39.789710 29764 raft_consensus.cc:729] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6aad75ee27140b6bb9c6838878682b7, State: Initialized, Role: FOLLOWER
I20260810 18:57:39.789953 29760 ts_tablet_manager.cc:1428] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260810 18:57:39.790141 29764 consensus_queue.cc:260] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.790812 29294 heartbeater.cc:503] Master 127.30.153.190:43463 was elected leader, sending a full tablet report...
I20260810 18:57:39.791152 29764 ts_tablet_manager.cc:1428] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260810 18:57:39.817917 29295 tablet.cc:2369] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:57:39.957928 29770 raft_consensus.cc:482] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 18:57:39.958191 29770 raft_consensus.cc:504] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.959535 29770 leader_election.cc:288] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6aad75ee27140b6bb9c6838878682b7 (127.30.153.129:45065), 8dee758b94dd4b3e84f6a5fe4a7dc386 (127.30.153.131:35289)
W20260810 18:57:39.962059 29734 tablet.cc:2369] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:57:39.964035 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.960090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:59549 (local address 127.30.153.129:45065)
0810 18:57:39.960237 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.960241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.960583 (+ 342us) server_negotiation.cc:407] Connection header received
0810 18:57:39.960750 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.960754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.960814 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.960907 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:39.961193 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.961689 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.963004 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.963401 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.963538 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.963591 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.963830 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.963907 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 18:57:39.964053 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.959908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45065 (local address 127.30.153.130:59549)
0810 18:57:39.960446 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:39.960458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:39.960623 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:39.960922 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:39.960925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:39.960941 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:39.961064 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.961069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.961813 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.961816 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.962858 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.962867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.963536 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.963540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.963692 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.963711 (+ 19us) client_negotiation.cc:769] Sending connection context
0810 18:57:39.963810 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 18:57:39.963884 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":132,"thread_start_us":141,"threads_started":1}
I20260810 18:57:39.964133 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.960182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.153.130:50431 (local address 127.30.153.131:35289)
0810 18:57:39.960343 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.960351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.960370 (+ 19us) server_negotiation.cc:407] Connection header received
0810 18:57:39.960436 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.960438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.960480 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.960579 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 18:57:39.961029 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.961564 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.963047 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.963464 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.963612 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.963685 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:39.963922 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 18:57:39.963997 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 18:57:39.964184 29503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.959908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.131:35289 (local address 127.30.153.130:50431)
0810 18:57:39.960038 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:39.960069 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 18:57:39.960248 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:57:39.960616 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:57:39.960620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:57:39.960640 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 18:57:39.960851 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.960864 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.961728 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.961733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.962909 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:57:39.962919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.963612 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.963616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:57:39.963789 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.963807 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 18:57:39.963903 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:57:39.963979 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":45}
I20260810 18:57:39.964815 29223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0359679582eb49749cde5a84a87fbf87" candidate_uuid: "bae84caa79704668bf91d3f9d769eac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6aad75ee27140b6bb9c6838878682b7" is_pre_election: true
I20260810 18:57:39.964831 29649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0359679582eb49749cde5a84a87fbf87" candidate_uuid: "bae84caa79704668bf91d3f9d769eac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" is_pre_election: true
I20260810 18:57:39.965189 29649 raft_consensus.cc:2454] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bae84caa79704668bf91d3f9d769eac9 in term 0.
I20260810 18:57:39.965189 29223 raft_consensus.cc:2454] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bae84caa79704668bf91d3f9d769eac9 in term 0.
I20260810 18:57:39.965902 29327 leader_election.cc:302] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dee758b94dd4b3e84f6a5fe4a7dc386, bae84caa79704668bf91d3f9d769eac9; no voters:
I20260810 18:57:39.966238 29770 raft_consensus.cc:2781] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 18:57:39.966336 29770 raft_consensus.cc:482] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 18:57:39.966382 29770 raft_consensus.cc:3037] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:39.967515 29770 raft_consensus.cc:504] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.967878 29770 leader_election.cc:288] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [CANDIDATE]: Term 1 election: Requested vote from peers f6aad75ee27140b6bb9c6838878682b7 (127.30.153.129:45065), 8dee758b94dd4b3e84f6a5fe4a7dc386 (127.30.153.131:35289)
I20260810 18:57:39.968256 29223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0359679582eb49749cde5a84a87fbf87" candidate_uuid: "bae84caa79704668bf91d3f9d769eac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6aad75ee27140b6bb9c6838878682b7"
I20260810 18:57:39.968287 29649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0359679582eb49749cde5a84a87fbf87" candidate_uuid: "bae84caa79704668bf91d3f9d769eac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386"
I20260810 18:57:39.968410 29223 raft_consensus.cc:3037] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:39.968426 29649 raft_consensus.cc:3037] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:39.969478 29649 raft_consensus.cc:2454] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bae84caa79704668bf91d3f9d769eac9 in term 1.
I20260810 18:57:39.969527 29223 raft_consensus.cc:2454] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bae84caa79704668bf91d3f9d769eac9 in term 1.
I20260810 18:57:39.969834 29327 leader_election.cc:302] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8dee758b94dd4b3e84f6a5fe4a7dc386, bae84caa79704668bf91d3f9d769eac9; no voters:
I20260810 18:57:39.970022 29770 raft_consensus.cc:2781] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:39.970244 29770 raft_consensus.cc:686] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 1 LEADER]: Becoming Leader. State: Replica: bae84caa79704668bf91d3f9d769eac9, State: Running, Role: LEADER
I20260810 18:57:39.970628 29770 consensus_queue.cc:237] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } }
I20260810 18:57:39.974211 28994 catalog_manager.cc:5495] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bae84caa79704668bf91d3f9d769eac9 (127.30.153.130). New cstate: current_term: 1 leader_uuid: "bae84caa79704668bf91d3f9d769eac9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bae84caa79704668bf91d3f9d769eac9" member_type: VOTER last_known_addr { host: "127.30.153.130" port: 41413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 } health_report { overall_health: UNKNOWN } } }
I20260810 18:57:40.000744 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:39.996711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50274 (local address 127.30.153.130:41413)
0810 18:57:39.996842 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:39.996846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:39.996861 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:57:39.997019 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:39.997022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:39.997081 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:39.997175 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:57:39.997503 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.998027 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.999154 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:39.999376 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:39.999464 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:39.999794 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:57:40.000233 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:57:40.000331 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:40.000552 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 18:57:40.000621 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:57:40.008288 29649 raft_consensus.cc:1261] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 1 FOLLOWER]: Refusing update from remote peer bae84caa79704668bf91d3f9d769eac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:40.008288 29223 raft_consensus.cc:1261] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 1 FOLLOWER]: Refusing update from remote peer bae84caa79704668bf91d3f9d769eac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:57:40.009279 29770 consensus_queue.cc:1035] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dee758b94dd4b3e84f6a5fe4a7dc386" member_type: VOTER last_known_addr { host: "127.30.153.131" port: 35289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:57:40.009620 29773 consensus_queue.cc:1035] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6aad75ee27140b6bb9c6838878682b7" member_type: VOTER last_known_addr { host: "127.30.153.129" port: 45065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 18:57:40.016528 29502 tablet.cc:2369] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 18:57:40.018716 29780 mvcc.cc:205] Tried to move back new op lower bound from 7317046312984862720 to 7317046312842289152. Current Snapshot: MvccSnapshot[applied={T|T < 7317046312984862720}]
I20260810 18:57:40.027994 28993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:40.028195 28993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 18:57:40.030782 28993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ce37f23a26604d0da55eeb919fd3a11d: Sending DeleteTablet for 3 replicas of tablet 0359679582eb49749cde5a84a87fbf87
I20260810 18:57:40.031659 29392 tablet_service.cc:1514] Processing DeleteTablet for tablet 0359679582eb49749cde5a84a87fbf87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:40 UTC) from {username='slave'} at 127.0.0.1:50264
I20260810 18:57:40.031635 29192 tablet_service.cc:1514] Processing DeleteTablet for tablet 0359679582eb49749cde5a84a87fbf87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:40 UTC) from {username='slave'} at 127.0.0.1:41516
I20260810 18:57:40.031893 29623 tablet_service.cc:1514] Processing DeleteTablet for tablet 0359679582eb49749cde5a84a87fbf87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 18:57:40 UTC) from {username='slave'} at 127.0.0.1:41522
I20260810 18:57:40.032413 29794 tablet_replica.cc:330] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: stopping tablet replica
I20260810 18:57:40.032449 29791 tablet_replica.cc:330] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7: stopping tablet replica
I20260810 18:57:40.032776 29794 raft_consensus.cc:2229] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:40.032804 29791 raft_consensus.cc:2229] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 18:57:40.033136 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 29088
I20260810 18:57:40.033144 29794 raft_consensus.cc:2258] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:40.033166 29791 raft_consensus.cc:2258] T 0359679582eb49749cde5a84a87fbf87 P f6aad75ee27140b6bb9c6838878682b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:40.033950 29793 tablet_replica.cc:330] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: stopping tablet replica
I20260810 18:57:40.034291 29793 raft_consensus.cc:2229] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 18:57:40.034758 29793 raft_consensus.cc:2258] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 18:57:40.035140 29794 ts_tablet_manager.cc:1905] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:40.037997 29793 ts_tablet_manager.cc:1905] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 18:57:40.039086 29794 ts_tablet_manager.cc:1918] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:40.039214 29794 log.cc:1196] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-2/wal/wals/0359679582eb49749cde5a84a87fbf87
I20260810 18:57:40.039592 29794 ts_tablet_manager.cc:1939] T 0359679582eb49749cde5a84a87fbf87 P 8dee758b94dd4b3e84f6a5fe4a7dc386: Deleting consensus metadata
I20260810 18:57:40.041112 28975 catalog_manager.cc:4841] TS 8dee758b94dd4b3e84f6a5fe4a7dc386 (127.30.153.131:35289): tablet 0359679582eb49749cde5a84a87fbf87 (table test-table [id=5fbb5b46567b401ca8590e95439afaea]) successfully deleted
I20260810 18:57:40.040846 29793 ts_tablet_manager.cc:1918] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 18:57:40.041417 29793 log.cc:1196] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Deleting WAL directory at /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786387990686100-31334-0/minicluster-data/ts-1/wal/wals/0359679582eb49749cde5a84a87fbf87
I20260810 18:57:40.041934 29793 ts_tablet_manager.cc:1939] T 0359679582eb49749cde5a84a87fbf87 P bae84caa79704668bf91d3f9d769eac9: Deleting consensus metadata
I20260810 18:57:40.043418 28978 catalog_manager.cc:4841] TS bae84caa79704668bf91d3f9d769eac9 (127.30.153.130:41413): tablet 0359679582eb49749cde5a84a87fbf87 (table test-table [id=5fbb5b46567b401ca8590e95439afaea]) successfully deleted
W20260810 18:57:40.046759 28978 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.153.129:45065 (error 108)
I20260810 18:57:40.047060 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 29298
I20260810 18:57:40.051194 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:40.050806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.129:45065 (local address 127.0.0.1:41518)
0810 18:57:40.050972 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 18:57:40.051079 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.129:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:57:40.051527 28978 catalog_manager.cc:4584] TS f6aad75ee27140b6bb9c6838878682b7 (127.30.153.129:45065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0359679582eb49749cde5a84a87fbf87: Network error: Client connection negotiation failed: client connection to 127.30.153.129:45065: connect: Connection refused (error 111)
I20260810 18:57:40.058310 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 29509
I20260810 18:57:40.073753 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 28949
2026-08-10T18:57:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1222 ms)
[----------] 6 tests from EncryptionPolicyTest (11753 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:40 dist-test-slave-68xk krb5kdc[29802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:40 dist-test-slave-68xk krb5kdc[29802](info): set up 2 sockets
Aug 10 18:57:40 dist-test-slave-68xk krb5kdc[29802](info): commencing operation
krb5kdc: starting...
W20260810 18:57:41.339316 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:41 dist-test-slave-68xk krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:41.468988 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:41.584597 29817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:41.584911 29817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:41.584985 29817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:41.589036 29817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:41.589145 29817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:41.589184 29817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:41.589221 29817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:41.589257 29817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:41.594544 29817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43195
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:41.595665 29817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:41.596740 29817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:41.605666 29823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:41.605633 29825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:41.605679 29822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:41.606119 29817 server_base.cc:1034] running on GCE node
I20260810 18:57:41.606735 29817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:41.607800 29817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:41.608970 29817 hybrid_clock.cc:648] HybridClock initialized: now 1786388261608952 us; error 30 us; skew 500 ppm
Aug 10 18:57:41 dist-test-slave-68xk krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388261, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:41.611922 29817 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:41.612903 29817 webserver.cc:460] Webserver started at http://127.0.0.1:35095/ using document root <none> and password file <none>
I20260810 18:57:41.613449 29817 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:41.613510 29817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:41.613683 29817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:41.615819 29817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "a1c776fc489d4f92a761d0e31d6746a0"
format_stamp: "Formatted at 2026-08-10 18:57:41 on dist-test-slave-68xk"
server_key: "7c113c668cc19bd1ab5cc744fcb1e27a"
server_key_iv: "6dec205711aadd38fe6df00dfa3487c5"
server_key_version: "encryptionkey@0"
I20260810 18:57:41.616350 29817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "a1c776fc489d4f92a761d0e31d6746a0"
format_stamp: "Formatted at 2026-08-10 18:57:41 on dist-test-slave-68xk"
server_key: "7c113c668cc19bd1ab5cc744fcb1e27a"
server_key_iv: "6dec205711aadd38fe6df00dfa3487c5"
server_key_version: "encryptionkey@0"
I20260810 18:57:41.620208 29817 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 18:57:41.622692 29839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:41.624037 29817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:57:41.624215 29817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "a1c776fc489d4f92a761d0e31d6746a0"
format_stamp: "Formatted at 2026-08-10 18:57:41 on dist-test-slave-68xk"
server_key: "7c113c668cc19bd1ab5cc744fcb1e27a"
server_key_iv: "6dec205711aadd38fe6df00dfa3487c5"
server_key_version: "encryptionkey@0"
I20260810 18:57:41.624312 29817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:41.637861 29817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:41.640969 29817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:41.641207 29817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:41.648820 29817 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43195
I20260810 18:57:41.648864 29925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43195 every 8 connection(s)
I20260810 18:57:41.649768 29817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:41.652493 29930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:41.655733 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 29817
I20260810 18:57:41.655916 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:41.656234 31334 external_mini_cluster.cc:1428] Setting key 563b164ca6ebb1fb8176ed6ed69bc850
I20260810 18:57:41.658579 29930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0: Bootstrap starting.
I20260810 18:57:41.661298 29930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:41.662171 29930 log.cc:824] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:41.664424 29930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0: No bootstrap required, opened a new log
Aug 10 18:57:41 dist-test-slave-68xk krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:57:41.668594 29930 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c776fc489d4f92a761d0e31d6746a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260810 18:57:41.668900 29930 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:41.668970 29930 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1c776fc489d4f92a761d0e31d6746a0, State: Initialized, Role: FOLLOWER
I20260810 18:57:41.669466 29930 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c776fc489d4f92a761d0e31d6746a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260810 18:57:41.669641 29930 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:41.669715 29930 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:41.669831 29930 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:41.671017 29930 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c776fc489d4f92a761d0e31d6746a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260810 18:57:41.671389 29930 leader_election.cc:302] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1c776fc489d4f92a761d0e31d6746a0; no voters:
I20260810 18:57:41.671794 29930 leader_election.cc:288] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:41.671986 29939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:41.672247 29939 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [term 1 LEADER]: Becoming Leader. State: Replica: a1c776fc489d4f92a761d0e31d6746a0, State: Running, Role: LEADER
I20260810 18:57:41.672556 29939 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c776fc489d4f92a761d0e31d6746a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } }
I20260810 18:57:41.673100 29930 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:41.673422 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:41.657690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55817 (local address 127.0.0.1:43195)
0810 18:57:41.658124 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:41.658143 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:41.658185 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:57:41.659025 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:41.659048 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:41.659460 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:41.659841 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:41.660490 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:41.662000 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:41.663176 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:41.663842 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:41.664006 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:41.666915 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:41.666922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:41.666940 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:41.666974 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:41.669630 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:41.670230 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:41.670235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:41.670241 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:41.670375 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:41.670745 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:41.670749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:41.670751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:41.671284 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:41.671504 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:41.671844 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 18:57:41.672275 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":115,"threads_started":1}
I20260810 18:57:41.674501 29943 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1c776fc489d4f92a761d0e31d6746a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c776fc489d4f92a761d0e31d6746a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } } }
I20260810 18:57:41.674723 29943 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:41.676029 29941 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1c776fc489d4f92a761d0e31d6746a0. Latest consensus state: current_term: 1 leader_uuid: "a1c776fc489d4f92a761d0e31d6746a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1c776fc489d4f92a761d0e31d6746a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43195 } } }
I20260810 18:57:41.676410 29941 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:41.677016 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 18:57:41.678122 29969 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:57:41.678202 29969 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:57:41.678272 29966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:41.679188 29966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:41.685645 29966 catalog_manager.cc:1347] Generated new cluster ID: f95de9a8bd1a407887dcc4c91efe2a5a
I20260810 18:57:41.685715 29966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:41.688023 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:41.679368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55091 (local address 127.0.0.1:43195)
0810 18:57:41.679556 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:41.679560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:41.679572 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:57:41.679613 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:41.679615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:41.679648 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:41.679739 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:41.680424 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:41.680950 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:41.681789 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:41.682039 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:41.682153 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:41.684189 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:41.684193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:41.684196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:41.684222 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:41.686023 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:41.686651 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:41.686655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:41.686657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:41.686750 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:41.687139 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:41.687142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:41.687143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:41.687336 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:41.687462 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:41.687772 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 18:57:41.687866 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260810 18:57:41.695561 29966 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:41.696753 29966 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:41.706827 29966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1c776fc489d4f92a761d0e31d6746a0: Generated new TSK 0
I20260810 18:57:41.707607 29966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:41.715284 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 29817
W20260810 18:57:41.728890 29797 connection.cc:535] client connection to 127.0.0.1:43195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43195): Connection reset by peer (error 104)
2026-08-10T18:57:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:41 dist-test-slave-68xk krb5kdc[29989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:41 dist-test-slave-68xk krb5kdc[29989](info): set up 2 sockets
Aug 10 18:57:41 dist-test-slave-68xk krb5kdc[29989](info): commencing operation
krb5kdc: starting...
W20260810 18:57:42.987748 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:43 dist-test-slave-68xk krb5kdc[29989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:43.123915 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38171
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:43.243041 30014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:43.243300 30014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:43.243376 30014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:43.247387 30014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:43.247442 30014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:43.247462 30014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:43.247478 30014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:43.247541 30014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:43.253094 30014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38171
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:43.254143 30014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:43.255192 30014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:43.263770 30023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:43.263803 30021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:43.263810 30025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:43.264185 30014 server_base.cc:1034] running on GCE node
I20260810 18:57:43.264773 30014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:43.265987 30014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:43.267154 30014 hybrid_clock.cc:648] HybridClock initialized: now 1786388263267128 us; error 47 us; skew 500 ppm
Aug 10 18:57:43 dist-test-slave-68xk krb5kdc[29989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:43.274718 30014 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:43.275750 30014 webserver.cc:460] Webserver started at http://127.0.0.1:34155/ using document root <none> and password file <none>
I20260810 18:57:43.276381 30014 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:43.276448 30014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:43.276679 30014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:43.278898 30014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "4f7bde7142c24fb980aa30f64b9f38cb"
format_stamp: "Formatted at 2026-08-10 18:57:43 on dist-test-slave-68xk"
server_key: "b0189ec03f7f273b029e5a36c3ad41d8"
server_key_iv: "21e912b667607e7753fe7ba12ce255ff"
server_key_version: "encryptionkey@0"
I20260810 18:57:43.279521 30014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "4f7bde7142c24fb980aa30f64b9f38cb"
format_stamp: "Formatted at 2026-08-10 18:57:43 on dist-test-slave-68xk"
server_key: "b0189ec03f7f273b029e5a36c3ad41d8"
server_key_iv: "21e912b667607e7753fe7ba12ce255ff"
server_key_version: "encryptionkey@0"
I20260810 18:57:43.283823 30014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:43.286614 30041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:43.287956 30014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:43.288125 30014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "4f7bde7142c24fb980aa30f64b9f38cb"
format_stamp: "Formatted at 2026-08-10 18:57:43 on dist-test-slave-68xk"
server_key: "b0189ec03f7f273b029e5a36c3ad41d8"
server_key_iv: "21e912b667607e7753fe7ba12ce255ff"
server_key_version: "encryptionkey@0"
I20260810 18:57:43.288259 30014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:43.304906 30014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:43.308538 30014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:43.308748 30014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:43.316077 30014 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38171
I20260810 18:57:43.316105 30121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38171 every 8 connection(s)
I20260810 18:57:43.317036 30014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:43.319912 30124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:43.320878 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 30014
I20260810 18:57:43.321069 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:43.321398 31334 external_mini_cluster.cc:1428] Setting key 9a32b4ea15550d1128b4701ce9876bf2
I20260810 18:57:43.326045 30124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb: Bootstrap starting.
I20260810 18:57:43.328423 30124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:43.329216 30124 log.cc:824] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:43.331496 30124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb: No bootstrap required, opened a new log
I20260810 18:57:43.331537 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:43.322957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47715 (local address 127.0.0.1:38171)
0810 18:57:43.323442 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:43.323458 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:43.323500 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:57:43.324253 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:43.324272 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:43.324641 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:43.325018 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:43.325480 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:43.326698 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:43.327733 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:43.328283 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:43.328482 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:43.329542 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:43.329548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:43.329561 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:43.329982 (+ 421us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:43.330052 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:43.330078 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:43.330256 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:43.330499 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 18:57:43.330664 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":134,"threads_started":1}
I20260810 18:57:43.334905 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:43.335680 30124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7bde7142c24fb980aa30f64b9f38cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } }
I20260810 18:57:43.335938 30124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:43.336023 30124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f7bde7142c24fb980aa30f64b9f38cb, State: Initialized, Role: FOLLOWER
I20260810 18:57:43.336526 30124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7bde7142c24fb980aa30f64b9f38cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } }
I20260810 18:57:43.336689 30124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:43.336771 30124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:43.336887 30124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:43.337882 30124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7bde7142c24fb980aa30f64b9f38cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } }
I20260810 18:57:43.338227 30124 leader_election.cc:302] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f7bde7142c24fb980aa30f64b9f38cb; no voters:
I20260810 18:57:43.338608 30124 leader_election.cc:288] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:43.338716 30139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:43.338954 30139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [term 1 LEADER]: Becoming Leader. State: Replica: 4f7bde7142c24fb980aa30f64b9f38cb, State: Running, Role: LEADER
I20260810 18:57:43.339249 30139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7bde7142c24fb980aa30f64b9f38cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } }
I20260810 18:57:43.339962 30124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:43.340422 30139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f7bde7142c24fb980aa30f64b9f38cb. Latest consensus state: current_term: 1 leader_uuid: "4f7bde7142c24fb980aa30f64b9f38cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7bde7142c24fb980aa30f64b9f38cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } } }
I20260810 18:57:43.340539 30139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:43.340917 30142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f7bde7142c24fb980aa30f64b9f38cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f7bde7142c24fb980aa30f64b9f38cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38171 } } }
I20260810 18:57:43.341020 30142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:43.341742 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:43.337108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41427 (local address 127.0.0.1:38171)
0810 18:57:43.337228 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:43.337232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:43.337397 (+ 165us) server_negotiation.cc:407] Connection header received
0810 18:57:43.337541 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:43.337544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:43.337588 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:43.337683 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:43.337956 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:43.338625 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:43.339802 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:43.340129 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:43.340264 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:43.341240 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:43.341243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:43.341245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:43.341258 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:43.341285 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:43.341298 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:43.341405 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:43.341582 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 18:57:43.341618 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 18:57:43.344903 30164 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:57:43.345000 30164 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:57:43.345345 30153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:43.346302 30153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:43.352182 30153 catalog_manager.cc:1347] Generated new cluster ID: aa63109bfc5740db9ba30421b4a19bcb
I20260810 18:57:43.352246 30153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:43.373845 30153 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:43.375166 30153 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:43.390457 30153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f7bde7142c24fb980aa30f64b9f38cb: Generated new TSK 0
I20260810 18:57:43.391295 30153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:43.400820 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 30014
W20260810 18:57:43.409945 29983 connection.cc:535] client connection to 127.0.0.1:38171 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38171): Connection reset by peer (error 104)
2026-08-10T18:57:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1681 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:43 dist-test-slave-68xk krb5kdc[30176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:43 dist-test-slave-68xk krb5kdc[30176](info): set up 2 sockets
Aug 10 18:57:43 dist-test-slave-68xk krb5kdc[30176](info): commencing operation
krb5kdc: starting...
W20260810 18:57:44.679678 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:44 dist-test-slave-68xk krb5kdc[30176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:44.810515 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:44.924832 30196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:44.925119 30196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:44.925164 30196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:44.929188 30196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:44.929250 30196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:44.929270 30196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:44.929287 30196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:44.929302 30196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:44.934626 30196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:44.935642 30196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:44.936638 30196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:44.945410 30206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:44.945418 30209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:44.945416 30205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:44.945428 30196 server_base.cc:1034] running on GCE node
I20260810 18:57:44.946257 30196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:44.947459 30196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:44.948623 30196 hybrid_clock.cc:648] HybridClock initialized: now 1786388264948597 us; error 45 us; skew 500 ppm
Aug 10 18:57:44 dist-test-slave-68xk krb5kdc[30176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388264, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:44.951434 30196 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:44.952342 30196 webserver.cc:460] Webserver started at http://127.0.0.1:38121/ using document root <none> and password file <none>
I20260810 18:57:44.952905 30196 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:44.952999 30196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:44.953246 30196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:44.955376 30196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "f3130c773d5843579341db342e368728"
format_stamp: "Formatted at 2026-08-10 18:57:44 on dist-test-slave-68xk"
server_key: "a91f9ce738a6f3826da1773e8d0feff9"
server_key_iv: "793fa4424cea8ba86203abc8aa5f9ed1"
server_key_version: "encryptionkey@0"
I20260810 18:57:44.955952 30196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "f3130c773d5843579341db342e368728"
format_stamp: "Formatted at 2026-08-10 18:57:44 on dist-test-slave-68xk"
server_key: "a91f9ce738a6f3826da1773e8d0feff9"
server_key_iv: "793fa4424cea8ba86203abc8aa5f9ed1"
server_key_version: "encryptionkey@0"
I20260810 18:57:44.959879 30196 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:44.962389 30218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:44.963692 30196 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:57:44.963838 30196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "f3130c773d5843579341db342e368728"
format_stamp: "Formatted at 2026-08-10 18:57:44 on dist-test-slave-68xk"
server_key: "a91f9ce738a6f3826da1773e8d0feff9"
server_key_iv: "793fa4424cea8ba86203abc8aa5f9ed1"
server_key_version: "encryptionkey@0"
I20260810 18:57:44.963953 30196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:44.968698 30196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:44.971676 30196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:44.971872 30196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:44.979310 30196 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44659
I20260810 18:57:44.979338 30303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44659 every 8 connection(s)
I20260810 18:57:44.980209 30196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:44.983089 30304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:44.987859 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 30196
I20260810 18:57:44.987996 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:44.988276 31334 external_mini_cluster.cc:1428] Setting key 8335b6cd128cd9a8478b5d14a725c5d3
I20260810 18:57:44.988791 30304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728: Bootstrap starting.
I20260810 18:57:44.991382 30304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:44.992297 30304 log.cc:824] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:44.993391 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:44.989647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37307 (local address 127.0.0.1:44659)
0810 18:57:44.990079 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:44.990094 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:44.990128 (+ 34us) server_negotiation.cc:407] Connection header received
0810 18:57:44.990820 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:44.990838 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:44.991110 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:44.991449 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:44.992251 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:44.992255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:44.992267 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:44.992278 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:44.992329 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:44.992347 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:44.992465 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:44.992570 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 18:57:44.992728 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":130,"threads_started":1}
I20260810 18:57:44.994302 30304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728: No bootstrap required, opened a new log
I20260810 18:57:44.996132 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:44.998694 30304 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3130c773d5843579341db342e368728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } }
I20260810 18:57:44.998970 30304 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:44.999032 30304 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3130c773d5843579341db342e368728, State: Initialized, Role: FOLLOWER
I20260810 18:57:44.999468 30304 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3130c773d5843579341db342e368728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } }
I20260810 18:57:44.999635 30304 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:44.999730 30304 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:44.999830 30304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:45.000310 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:44.998559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50697 (local address 127.0.0.1:44659)
0810 18:57:44.998687 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:44.998692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:44.998912 (+ 220us) server_negotiation.cc:407] Connection header received
0810 18:57:44.999003 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:44.999006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:44.999046 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:44.999148 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:44.999926 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:44.999929 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:44.999931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:44.999945 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:44.999975 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:44.999988 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:45.000094 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:45.000162 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 18:57:45.000201 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:57:45.000906 30304 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3130c773d5843579341db342e368728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } }
I20260810 18:57:45.001252 30304 leader_election.cc:302] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3130c773d5843579341db342e368728; no voters:
I20260810 18:57:45.001601 30304 leader_election.cc:288] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:45.001665 30317 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:45.001916 30317 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [term 1 LEADER]: Becoming Leader. State: Replica: f3130c773d5843579341db342e368728, State: Running, Role: LEADER
I20260810 18:57:45.002219 30317 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3130c773d5843579341db342e368728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } }
I20260810 18:57:45.002890 30304 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:45.003954 30320 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3130c773d5843579341db342e368728. Latest consensus state: current_term: 1 leader_uuid: "f3130c773d5843579341db342e368728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3130c773d5843579341db342e368728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } } }
I20260810 18:57:45.004101 30320 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:45.004504 30334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:45.004391 30318 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3130c773d5843579341db342e368728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3130c773d5843579341db342e368728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44659 } } }
I20260810 18:57:45.005496 30318 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:45.007776 30334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:45.014891 30334 catalog_manager.cc:1347] Generated new cluster ID: 11abd6a3c0204acc8e5c1fedc3a16129
I20260810 18:57:45.014974 30334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:45.035809 30334 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:45.036121 30334 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:45.046231 30334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3130c773d5843579341db342e368728: Generated new TSK 0
I20260810 18:57:45.047082 30334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:45.062666 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 30196
2026-08-10T18:57:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1664 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:45 dist-test-slave-68xk krb5kdc[30353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:45 dist-test-slave-68xk krb5kdc[30353](info): set up 2 sockets
Aug 10 18:57:45 dist-test-slave-68xk krb5kdc[30353](info): commencing operation
krb5kdc: starting...
W20260810 18:57:46.319422 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:46 dist-test-slave-68xk krb5kdc[30353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:46.450023 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33879
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:46.562104 30378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:46.562345 30378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:46.562386 30378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:46.566311 30378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:46.566366 30378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:46.566385 30378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:46.566401 30378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:46.566470 30378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:46.571524 30378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:46.572497 30378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:46.573421 30378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:46.582404 30388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:46.582430 30396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:46.582430 30387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:46.582899 30378 server_base.cc:1034] running on GCE node
I20260810 18:57:46.583451 30378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:46.584592 30378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:46.585798 30378 hybrid_clock.cc:648] HybridClock initialized: now 1786388266585777 us; error 32 us; skew 500 ppm
Aug 10 18:57:46 dist-test-slave-68xk krb5kdc[30353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388266, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:46.593196 30378 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:46.594198 30378 webserver.cc:460] Webserver started at http://127.0.0.1:42879/ using document root <none> and password file <none>
I20260810 18:57:46.594872 30378 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:46.594974 30378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:46.595216 30378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:46.597353 30378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "97a448805fa34020a89ba1235ee5be06"
format_stamp: "Formatted at 2026-08-10 18:57:46 on dist-test-slave-68xk"
server_key: "b5fbc30b486b4329c4229d1f9b37e00e"
server_key_iv: "9fcafc0a2afbbe1303f12d3cdb2f1f0c"
server_key_version: "encryptionkey@0"
I20260810 18:57:46.597976 30378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "97a448805fa34020a89ba1235ee5be06"
format_stamp: "Formatted at 2026-08-10 18:57:46 on dist-test-slave-68xk"
server_key: "b5fbc30b486b4329c4229d1f9b37e00e"
server_key_iv: "9fcafc0a2afbbe1303f12d3cdb2f1f0c"
server_key_version: "encryptionkey@0"
I20260810 18:57:46.601994 30378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:57:46.604647 30409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:46.605957 30378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 18:57:46.606106 30378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "97a448805fa34020a89ba1235ee5be06"
format_stamp: "Formatted at 2026-08-10 18:57:46 on dist-test-slave-68xk"
server_key: "b5fbc30b486b4329c4229d1f9b37e00e"
server_key_iv: "9fcafc0a2afbbe1303f12d3cdb2f1f0c"
server_key_version: "encryptionkey@0"
I20260810 18:57:46.606227 30378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:46.626973 30378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:46.630344 30378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:46.630582 30378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:46.638477 30378 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33879
I20260810 18:57:46.638507 30489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33879 every 8 connection(s)
I20260810 18:57:46.639469 30378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:46.642342 30490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:46.647003 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 30378
I20260810 18:57:46.647162 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:46.647492 31334 external_mini_cluster.cc:1428] Setting key 9fd1e92162416903ee08b735b11dca24
I20260810 18:57:46.648332 30490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06: Bootstrap starting.
I20260810 18:57:46.650941 30490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:46.651882 30490 log.cc:824] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:46.653810 30490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06: No bootstrap required, opened a new log
Aug 10 18:57:46 dist-test-slave-68xk krb5kdc[30353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:57:46.657953 30490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a448805fa34020a89ba1235ee5be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33879 } }
I20260810 18:57:46.658183 30490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:46.658267 30490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97a448805fa34020a89ba1235ee5be06, State: Initialized, Role: FOLLOWER
I20260810 18:57:46.658792 30490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a448805fa34020a89ba1235ee5be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33879 } }
I20260810 18:57:46.658918 30490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:46.659001 30490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:46.659125 30490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:46.660192 30490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a448805fa34020a89ba1235ee5be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33879 } }
I20260810 18:57:46.660594 30490 leader_election.cc:302] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97a448805fa34020a89ba1235ee5be06; no voters:
I20260810 18:57:46.660936 30490 leader_election.cc:288] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:46.661042 30500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:46.661269 30500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [term 1 LEADER]: Becoming Leader. State: Replica: 97a448805fa34020a89ba1235ee5be06, State: Running, Role: LEADER
I20260810 18:57:46.661590 30500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a448805fa34020a89ba1235ee5be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33879 } }
I20260810 18:57:46.662298 30490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:46.663247 30501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97a448805fa34020a89ba1235ee5be06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a448805fa34020a89ba1235ee5be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33879 } } }
I20260810 18:57:46.663369 30501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:46.663596 30501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97a448805fa34020a89ba1235ee5be06. Latest consensus state: current_term: 1 leader_uuid: "97a448805fa34020a89ba1235ee5be06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97a448805fa34020a89ba1235ee5be06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33879 } } }
I20260810 18:57:46.663676 30501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:46.663769 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:46.648956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36015 (local address 127.0.0.1:33879)
0810 18:57:46.649415 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:46.649429 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:46.649464 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:57:46.650094 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:46.650111 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:46.650542 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:46.650950 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:46.652071 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:46.653060 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:46.653943 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:46.654490 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:46.654665 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:46.657588 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:46.657593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:46.657606 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:46.657632 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:46.659695 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:46.660254 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:46.660259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:46.660264 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:46.660436 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:46.660774 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:46.660777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:46.660778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:46.661401 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:46.662189 (+ 788us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:46.662349 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 18:57:46.662688 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":124,"threads_started":1}
I20260810 18:57:46.664373 30518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:46.666955 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:46.667519 30518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:46.674045 30518 catalog_manager.cc:1347] Generated new cluster ID: 22965583908643ea81909af03b8eaf32
I20260810 18:57:46.674111 30518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:46.678385 30498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:46.669759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44789 (local address 127.0.0.1:33879)
0810 18:57:46.669875 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:46.669882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:46.669896 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:57:46.669932 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:46.669934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:46.669972 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:46.670068 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:46.670825 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:46.671366 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:46.672537 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:46.672976 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:46.673110 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:46.674941 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:46.674946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:46.674948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:46.674974 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:46.676637 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:46.677137 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:46.677140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:46.677141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:46.677220 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:46.677541 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:46.677544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:46.677545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:46.677734 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:46.677851 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:46.678122 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 18:57:46.678230 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:57:46.684445 30518 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:46.685385 30518 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:46.694288 30518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97a448805fa34020a89ba1235ee5be06: Generated new TSK 0
I20260810 18:57:46.695091 30518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:46.702405 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 30378
2026-08-10T18:57:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1646 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:46 dist-test-slave-68xk krb5kdc[30553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:46 dist-test-slave-68xk krb5kdc[30553](info): set up 2 sockets
Aug 10 18:57:46 dist-test-slave-68xk krb5kdc[30553](info): commencing operation
krb5kdc: starting...
W20260810 18:57:47.981324 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:48 dist-test-slave-68xk krb5kdc[30553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:48.118039 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:48.244377 30574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:48.244715 30574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:48.244812 30574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:48.249260 30574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:48.249377 30574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:48.249418 30574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:48.249456 30574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:48.249496 30574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:48.255131 30574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:48.256309 30574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:48.257406 30574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:48.266500 30587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:48.266588 30574 server_base.cc:1034] running on GCE node
W20260810 18:57:48.266500 30582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:48.266525 30583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:48.267316 30574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:48.268456 30574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:48.269680 30574 hybrid_clock.cc:648] HybridClock initialized: now 1786388268269645 us; error 49 us; skew 500 ppm
Aug 10 18:57:48 dist-test-slave-68xk krb5kdc[30553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388268, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:48.272729 30574 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:48.273650 30574 webserver.cc:460] Webserver started at http://127.0.0.1:32779/ using document root <none> and password file <none>
I20260810 18:57:48.274258 30574 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:48.274358 30574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:48.274705 30574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:48.276895 30574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "de4e4146bec74df5868034990106c368"
format_stamp: "Formatted at 2026-08-10 18:57:48 on dist-test-slave-68xk"
server_key: "13cebe8db547d2662fecd11d86416b21"
server_key_iv: "b21350d64a4aec2ac5ead385497e0fd9"
server_key_version: "encryptionkey@0"
I20260810 18:57:48.277472 30574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "de4e4146bec74df5868034990106c368"
format_stamp: "Formatted at 2026-08-10 18:57:48 on dist-test-slave-68xk"
server_key: "13cebe8db547d2662fecd11d86416b21"
server_key_iv: "b21350d64a4aec2ac5ead385497e0fd9"
server_key_version: "encryptionkey@0"
I20260810 18:57:48.281322 30574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 18:57:48.283957 30601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:48.285410 30574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 18:57:48.285560 30574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "de4e4146bec74df5868034990106c368"
format_stamp: "Formatted at 2026-08-10 18:57:48 on dist-test-slave-68xk"
server_key: "13cebe8db547d2662fecd11d86416b21"
server_key_iv: "b21350d64a4aec2ac5ead385497e0fd9"
server_key_version: "encryptionkey@0"
I20260810 18:57:48.285677 30574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:48.303128 30574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:48.306470 30574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:48.306706 30574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:48.314869 30574 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35649
I20260810 18:57:48.314900 30696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35649 every 8 connection(s)
I20260810 18:57:48.316008 30574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:48.318867 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:48.324637 30697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368: Bootstrap starting.
I20260810 18:57:48.325814 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 30574
I20260810 18:57:48.325990 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:48.326290 31334 external_mini_cluster.cc:1428] Setting key 39e494a79f6df84c05c6fb37ac6b410b
I20260810 18:57:48.327617 30697 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:48.328581 30697 log.cc:824] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:48.330904 30697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368: No bootstrap required, opened a new log
I20260810 18:57:48.335019 30697 raft_consensus.cc:348] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4e4146bec74df5868034990106c368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35649 } }
I20260810 18:57:48.335318 30697 raft_consensus.cc:374] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:48.335417 30697 raft_consensus.cc:729] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4e4146bec74df5868034990106c368, State: Initialized, Role: FOLLOWER
I20260810 18:57:48.335978 30697 consensus_queue.cc:260] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4e4146bec74df5868034990106c368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35649 } }
I20260810 18:57:48.336150 30697 raft_consensus.cc:388] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:48.336251 30697 raft_consensus.cc:482] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:48.336377 30697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:48.336848 30700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:48.327539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52851 (local address 127.0.0.1:35649)
0810 18:57:48.328095 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:48.328115 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:48.328160 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:57:48.329076 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:48.329097 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:48.329484 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:48.329859 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:48.330272 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:48.331755 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:48.332714 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:48.333445 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:48.333636 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:48.334569 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:48.334575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:48.334591 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:48.335094 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:48.335161 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:48.335192 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:48.335296 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:48.335599 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 18:57:48.335834 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":186,"threads_started":1}
I20260810 18:57:48.337495 30697 raft_consensus.cc:504] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4e4146bec74df5868034990106c368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35649 } }
I20260810 18:57:48.337846 30697 leader_election.cc:302] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de4e4146bec74df5868034990106c368; no voters:
I20260810 18:57:48.338240 30697 leader_election.cc:288] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:48.338321 30704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:48.338562 30704 raft_consensus.cc:686] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [term 1 LEADER]: Becoming Leader. State: Replica: de4e4146bec74df5868034990106c368, State: Running, Role: LEADER
I20260810 18:57:48.338868 30704 consensus_queue.cc:237] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4e4146bec74df5868034990106c368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35649 } }
I20260810 18:57:48.339627 30697 sys_catalog.cc:564] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:48.340350 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:48.340525 30710 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de4e4146bec74df5868034990106c368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4e4146bec74df5868034990106c368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35649 } } }
I20260810 18:57:48.340653 30710 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:48.341058 30707 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de4e4146bec74df5868034990106c368. Latest consensus state: current_term: 1 leader_uuid: "de4e4146bec74df5868034990106c368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4e4146bec74df5868034990106c368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35649 } } }
I20260810 18:57:48.341163 30707 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368 [sys.catalog]: This master's current role is: LEADER
W20260810 18:57:48.345027 30735 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:57:48.345136 30735 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:57:48.345273 30724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:48.346114 30724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:48.350214 30700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:48.343439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60735 (local address 127.0.0.1:35649)
0810 18:57:48.343550 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:48.343555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:48.344227 (+ 672us) server_negotiation.cc:407] Connection header received
0810 18:57:48.344325 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:48.344328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:48.344376 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:48.344467 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:48.344787 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:48.345666 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:48.346819 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:48.347189 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:48.347327 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:48.349776 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:48.349778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:48.349780 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:48.349792 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:48.349817 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:48.349827 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:48.349910 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:48.350009 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 18:57:48.350061 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260810 18:57:48.351714 30724 catalog_manager.cc:1347] Generated new cluster ID: 282fcc6dad50463abe99deb03bee1915
I20260810 18:57:48.351778 30724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:48.365808 30724 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:48.366807 30724 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:48.370975 30724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de4e4146bec74df5868034990106c368: Generated new TSK 0
I20260810 18:57:48.371611 30724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:48.380769 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 30574
2026-08-10T18:57:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1675 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:48 dist-test-slave-68xk krb5kdc[30751](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:48 dist-test-slave-68xk krb5kdc[30751](info): set up 2 sockets
Aug 10 18:57:48 dist-test-slave-68xk krb5kdc[30751](info): commencing operation
krb5kdc: starting...
W20260810 18:57:49.653182 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:49 dist-test-slave-68xk krb5kdc[30751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:49.790679 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:49.908268 30768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:49.908592 30768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:49.908675 30768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:49.912693 30768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:49.912794 30768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:49.912834 30768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:49.912868 30768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:49.912897 30768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:49.918175 30768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:49.919339 30768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:49.920469 30768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:49.929759 30781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:49.929831 30777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:49.929797 30779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:49.929984 30768 server_base.cc:1034] running on GCE node
I20260810 18:57:49.930742 30768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:49.931972 30768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:49.933190 30768 hybrid_clock.cc:648] HybridClock initialized: now 1786388269933159 us; error 46 us; skew 500 ppm
Aug 10 18:57:49 dist-test-slave-68xk krb5kdc[30751](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388269, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:49.936151 30768 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:49.937063 30768 webserver.cc:460] Webserver started at http://127.0.0.1:40553/ using document root <none> and password file <none>
I20260810 18:57:49.937644 30768 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:49.937745 30768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:49.937988 30768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:49.940146 30768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "d7fe09abd49b4038b16cf48161615760"
format_stamp: "Formatted at 2026-08-10 18:57:49 on dist-test-slave-68xk"
server_key: "e9834446c99332570c28e7e1a6b47426"
server_key_iv: "2da67ad4162a7516ec39bb8f814562be"
server_key_version: "encryptionkey@0"
I20260810 18:57:49.940739 30768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "d7fe09abd49b4038b16cf48161615760"
format_stamp: "Formatted at 2026-08-10 18:57:49 on dist-test-slave-68xk"
server_key: "e9834446c99332570c28e7e1a6b47426"
server_key_iv: "2da67ad4162a7516ec39bb8f814562be"
server_key_version: "encryptionkey@0"
I20260810 18:57:49.944525 30768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 18:57:49.947170 30795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:49.948398 30768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 18:57:49.948542 30768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "d7fe09abd49b4038b16cf48161615760"
format_stamp: "Formatted at 2026-08-10 18:57:49 on dist-test-slave-68xk"
server_key: "e9834446c99332570c28e7e1a6b47426"
server_key_iv: "2da67ad4162a7516ec39bb8f814562be"
server_key_version: "encryptionkey@0"
I20260810 18:57:49.948652 30768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:49.953348 30768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:49.956784 30768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:49.956988 30768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:49.964208 30768 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44691
I20260810 18:57:49.964295 30876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44691 every 8 connection(s)
I20260810 18:57:49.965186 30768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:49.968045 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:49.968284 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 30768
I20260810 18:57:49.968451 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:49.968765 31334 external_mini_cluster.cc:1428] Setting key c3a96e6ce3b9187d2602cdcb8c9e5e0c
I20260810 18:57:49.974393 30889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:49.970074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34869 (local address 127.0.0.1:44691)
0810 18:57:49.970516 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:49.970536 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:49.970579 (+ 43us) server_negotiation.cc:407] Connection header received
0810 18:57:49.971445 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:49.971469 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:49.971830 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:49.972196 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:49.972945 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:49.972951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:49.972966 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:49.972978 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:49.973042 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:49.973070 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:49.973212 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:49.973335 (+ 123us) server_negotiation.cc:299] Negotiation successful
0810 18:57:49.973488 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":105,"threads_started":1}
I20260810 18:57:49.975504 30883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760: Bootstrap starting.
I20260810 18:57:49.977245 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:49.977969 30883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:49.978802 30883 log.cc:824] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:49.981113 30889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:49.979404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56337 (local address 127.0.0.1:44691)
0810 18:57:49.979551 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:49.979555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:49.979707 (+ 152us) server_negotiation.cc:407] Connection header received
0810 18:57:49.979814 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:49.979816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:49.979848 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:49.979926 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:49.980677 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:49.980680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:49.980681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:49.980692 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:49.980717 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:49.980727 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:49.980800 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:49.980932 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 18:57:49.980965 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 18:57:49.981190 30883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760: No bootstrap required, opened a new log
I20260810 18:57:49.984962 30883 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fe09abd49b4038b16cf48161615760" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } }
I20260810 18:57:49.985226 30883 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:49.985321 30883 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7fe09abd49b4038b16cf48161615760, State: Initialized, Role: FOLLOWER
I20260810 18:57:49.985754 30883 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fe09abd49b4038b16cf48161615760" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } }
I20260810 18:57:49.985904 30883 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:49.985993 30883 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:49.986094 30883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:49.987119 30883 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fe09abd49b4038b16cf48161615760" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } }
I20260810 18:57:49.987478 30883 leader_election.cc:302] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7fe09abd49b4038b16cf48161615760; no voters:
I20260810 18:57:49.987833 30883 leader_election.cc:288] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:49.987980 30901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:49.988289 30901 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [term 1 LEADER]: Becoming Leader. State: Replica: d7fe09abd49b4038b16cf48161615760, State: Running, Role: LEADER
I20260810 18:57:49.988631 30901 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fe09abd49b4038b16cf48161615760" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } }
I20260810 18:57:49.989090 30883 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:49.990355 30903 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7fe09abd49b4038b16cf48161615760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fe09abd49b4038b16cf48161615760" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } } }
I20260810 18:57:49.990513 30903 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:49.990855 30904 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7fe09abd49b4038b16cf48161615760. Latest consensus state: current_term: 1 leader_uuid: "d7fe09abd49b4038b16cf48161615760" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fe09abd49b4038b16cf48161615760" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44691 } } }
I20260810 18:57:49.990937 30915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:49.991041 30904 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:49.994019 30915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:49.999315 30915 catalog_manager.cc:1347] Generated new cluster ID: e27f24f1cff5479e9ccf8f2b63ad55e0
I20260810 18:57:49.999385 30915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:50.015360 30915 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:50.015765 30915 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:50.027158 30915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7fe09abd49b4038b16cf48161615760: Generated new TSK 0
I20260810 18:57:50.027906 30915 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:50.041926 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 30768
2026-08-10T18:57:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1661 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:50 dist-test-slave-68xk krb5kdc[30931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:50 dist-test-slave-68xk krb5kdc[30931](info): set up 2 sockets
Aug 10 18:57:50 dist-test-slave-68xk krb5kdc[30931](info): commencing operation
krb5kdc: starting...
W20260810 18:57:51.339278 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.267s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:51 dist-test-slave-68xk krb5kdc[30931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:51.473287 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38105
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:51.586282 30952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:51.586633 30952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:51.586725 30952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:51.591171 30952 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:51.591297 30952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:51.591346 30952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:51.591410 30952 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:51.591447 30952 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:51.596824 30952 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:51.597996 30952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:51.599130 30952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:51.608559 30962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:51.608561 30964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:51.608561 30961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:51.608845 30952 server_base.cc:1034] running on GCE node
I20260810 18:57:51.609513 30952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:51.610735 30952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:51.611904 30952 hybrid_clock.cc:648] HybridClock initialized: now 1786388271611878 us; error 43 us; skew 500 ppm
Aug 10 18:57:51 dist-test-slave-68xk krb5kdc[30931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388271, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:51.621104 30952 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:51.622108 30952 webserver.cc:460] Webserver started at http://127.0.0.1:35303/ using document root <none> and password file <none>
I20260810 18:57:51.622776 30952 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:51.622843 30952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:51.623111 30952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:51.625214 30952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "2d3ecc5298904a19a9c722269e2685e8"
format_stamp: "Formatted at 2026-08-10 18:57:51 on dist-test-slave-68xk"
server_key: "fe7a852589583cc5da5b24b5f1b39d36"
server_key_iv: "530d60dd57031e698fc4bb6f3dfa6bbf"
server_key_version: "encryptionkey@0"
I20260810 18:57:51.625792 30952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "2d3ecc5298904a19a9c722269e2685e8"
format_stamp: "Formatted at 2026-08-10 18:57:51 on dist-test-slave-68xk"
server_key: "fe7a852589583cc5da5b24b5f1b39d36"
server_key_iv: "530d60dd57031e698fc4bb6f3dfa6bbf"
server_key_version: "encryptionkey@0"
I20260810 18:57:51.629550 30952 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260810 18:57:51.632004 30978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:51.633199 30952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:57:51.633343 30952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "2d3ecc5298904a19a9c722269e2685e8"
format_stamp: "Formatted at 2026-08-10 18:57:51 on dist-test-slave-68xk"
server_key: "fe7a852589583cc5da5b24b5f1b39d36"
server_key_iv: "530d60dd57031e698fc4bb6f3dfa6bbf"
server_key_version: "encryptionkey@0"
I20260810 18:57:51.633452 30952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:51.647596 30952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:51.650986 30952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:51.651192 30952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:51.658646 30952 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38105
I20260810 18:57:51.658675 31071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38105 every 8 connection(s)
I20260810 18:57:51.659595 30952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:51.660662 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 30952
I20260810 18:57:51.660802 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:51.661089 31334 external_mini_cluster.cc:1428] Setting key d450af0fa37216eff0710e9fdb99b71c
I20260810 18:57:51.662757 31072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:51.669466 31072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8: Bootstrap starting.
Aug 10 18:57:51 dist-test-slave-68xk krb5kdc[30931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:57:51.672156 31072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:51.672955 31072 log.cc:824] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:51.675282 31072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8: No bootstrap required, opened a new log
I20260810 18:57:51.677443 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:51.662677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:54571 (local address 127.0.0.1:38105)
0810 18:57:51.663131 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:51.663152 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:51.663197 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:57:51.664018 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:51.664040 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:51.664470 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:51.664831 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:51.665571 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:51.667084 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:51.667956 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:51.668658 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:51.668827 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:51.671724 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:51.671730 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:51.671745 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:51.671786 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:51.673998 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:51.674685 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:51.674690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:51.674695 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:51.674818 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:51.675232 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:51.675235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:51.675237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:51.675755 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:51.675941 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:51.676296 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 18:57:51.676640 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":159,"threads_started":1}
I20260810 18:57:51.679399 31072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3ecc5298904a19a9c722269e2685e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } }
I20260810 18:57:51.679657 31072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:51.679718 31072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d3ecc5298904a19a9c722269e2685e8, State: Initialized, Role: FOLLOWER
I20260810 18:57:51.680183 31072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3ecc5298904a19a9c722269e2685e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } }
I20260810 18:57:51.680351 31072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:51.680428 31072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:51.680538 31072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:51.680660 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:51.681479 31072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3ecc5298904a19a9c722269e2685e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } }
I20260810 18:57:51.681766 31072 leader_election.cc:302] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d3ecc5298904a19a9c722269e2685e8; no voters:
I20260810 18:57:51.682083 31072 leader_election.cc:288] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:51.682214 31086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:51.682456 31086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [term 1 LEADER]: Becoming Leader. State: Replica: 2d3ecc5298904a19a9c722269e2685e8, State: Running, Role: LEADER
I20260810 18:57:51.682781 31086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3ecc5298904a19a9c722269e2685e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } }
I20260810 18:57:51.683370 31072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:51.684801 31089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d3ecc5298904a19a9c722269e2685e8. Latest consensus state: current_term: 1 leader_uuid: "2d3ecc5298904a19a9c722269e2685e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3ecc5298904a19a9c722269e2685e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } } }
I20260810 18:57:51.684937 31089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:51.685317 31088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d3ecc5298904a19a9c722269e2685e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d3ecc5298904a19a9c722269e2685e8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38105 } } }
I20260810 18:57:51.685424 31088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:51.685704 31105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:51.689677 31105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:51.692814 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:51.683105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:37007 (local address 127.0.0.1:38105)
0810 18:57:51.683223 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:51.683228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:51.683241 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:57:51.683275 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:51.683280 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:51.683331 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:51.683414 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:51.684348 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:51.685199 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:51.686636 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:51.686873 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:51.686964 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:51.688595 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:51.688600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:51.688605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:51.688634 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:51.690468 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:51.691228 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:51.691233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:51.691235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:51.691333 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:51.691774 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:51.691778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:51.691780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:51.692018 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:51.692172 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:51.692544 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 18:57:51.692655 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260810 18:57:51.696224 31105 catalog_manager.cc:1347] Generated new cluster ID: a983f88f8108440aa320055719096a68
I20260810 18:57:51.696302 31105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:51.711901 31105 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:51.713173 31105 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:51.721402 31105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d3ecc5298904a19a9c722269e2685e8: Generated new TSK 0
I20260810 18:57:51.722287 31105 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:51.731190 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 30952
W20260810 18:57:51.746634 30929 connection.cc:535] client connection to 127.0.0.1:38105 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38105): Connection reset by peer (error 104)
2026-08-10T18:57:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1688 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:51 dist-test-slave-68xk krb5kdc[31118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:51 dist-test-slave-68xk krb5kdc[31118](info): set up 2 sockets
Aug 10 18:57:51 dist-test-slave-68xk krb5kdc[31118](info): commencing operation
krb5kdc: starting...
W20260810 18:57:53.029997 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:53 dist-test-slave-68xk krb5kdc[31118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:53.158183 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42213
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:53.271240 31138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:53.271509 31138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:53.271551 31138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:53.275612 31138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:53.275671 31138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:53.275691 31138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:53.275708 31138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:53.275723 31138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:53.280910 31138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:53.282050 31138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:53.283138 31138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:53.291524 31152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:53.291553 31150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:53.291553 31147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:53.291913 31138 server_base.cc:1034] running on GCE node
I20260810 18:57:53.292423 31138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:53.293497 31138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:53.294659 31138 hybrid_clock.cc:648] HybridClock initialized: now 1786388273294635 us; error 41 us; skew 500 ppm
Aug 10 18:57:53 dist-test-slave-68xk krb5kdc[31118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:53.297791 31138 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:53.298846 31138 webserver.cc:460] Webserver started at http://127.0.0.1:35619/ using document root <none> and password file <none>
I20260810 18:57:53.299431 31138 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:53.299494 31138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:53.299715 31138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:53.301790 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "01643318a337467aa76fa1d896482902"
format_stamp: "Formatted at 2026-08-10 18:57:53 on dist-test-slave-68xk"
server_key: "e8ae095db3601e4eb4f7a1f9865f52f5"
server_key_iv: "bd03aff307dad66781cc7140231af62a"
server_key_version: "encryptionkey@0"
I20260810 18:57:53.302369 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "01643318a337467aa76fa1d896482902"
format_stamp: "Formatted at 2026-08-10 18:57:53 on dist-test-slave-68xk"
server_key: "e8ae095db3601e4eb4f7a1f9865f52f5"
server_key_iv: "bd03aff307dad66781cc7140231af62a"
server_key_version: "encryptionkey@0"
I20260810 18:57:53.306339 31138 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260810 18:57:53.308911 31162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:53.310137 31138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:57:53.310277 31138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "01643318a337467aa76fa1d896482902"
format_stamp: "Formatted at 2026-08-10 18:57:53 on dist-test-slave-68xk"
server_key: "e8ae095db3601e4eb4f7a1f9865f52f5"
server_key_iv: "bd03aff307dad66781cc7140231af62a"
server_key_version: "encryptionkey@0"
I20260810 18:57:53.310384 31138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:53.340533 31138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:53.343743 31138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:53.343968 31138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:53.352020 31138 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42213
I20260810 18:57:53.352056 31248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42213 every 8 connection(s)
I20260810 18:57:53.353020 31138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:53.355561 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31138
I20260810 18:57:53.355741 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:53.356096 31334 external_mini_cluster.cc:1428] Setting key c2842377994a34649edd8bd3ac7578df
I20260810 18:57:53.356549 31250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:53.363479 31250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902: Bootstrap starting.
I20260810 18:57:53.365648 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:53.357722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:50273 (local address 127.0.0.1:42213)
0810 18:57:53.358234 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:53.358250 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:53.358287 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:57:53.358997 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:53.359014 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:53.359362 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:53.359700 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:53.360134 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:53.361156 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:53.362096 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:53.362659 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:53.362835 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:53.363872 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:53.363877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:53.363890 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:53.364290 (+ 400us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:53.364346 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:53.364369 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:53.364516 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:53.364710 (+ 194us) server_negotiation.cc:299] Negotiation successful
0810 18:57:53.364856 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":169,"threads_started":1}
I20260810 18:57:53.366008 31250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:53.366871 31250 log.cc:824] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:53.368734 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:53.369208 31250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902: No bootstrap required, opened a new log
I20260810 18:57:53.373674 31250 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01643318a337467aa76fa1d896482902" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42213 } }
I20260810 18:57:53.373965 31250 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:53.374035 31250 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01643318a337467aa76fa1d896482902, State: Initialized, Role: FOLLOWER
I20260810 18:57:53.374579 31250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01643318a337467aa76fa1d896482902" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42213 } }
I20260810 18:57:53.374773 31250 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:53.374872 31250 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:53.375003 31250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:53.375428 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:53.370855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:48993 (local address 127.0.0.1:42213)
0810 18:57:53.370992 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:53.370996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:53.371248 (+ 252us) server_negotiation.cc:407] Connection header received
0810 18:57:53.371383 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:53.371385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:53.371423 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:53.371490 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:53.371880 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:53.372385 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:53.373520 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:53.373741 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:53.373814 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:53.374835 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:53.374839 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:53.374845 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:53.374860 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:53.374889 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:53.374900 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:53.374993 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:53.375234 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 18:57:53.375289 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:57:53.376221 31250 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01643318a337467aa76fa1d896482902" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42213 } }
I20260810 18:57:53.376575 31250 leader_election.cc:302] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01643318a337467aa76fa1d896482902; no voters:
I20260810 18:57:53.376845 31250 leader_election.cc:288] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:53.376994 31267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:53.377251 31267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [term 1 LEADER]: Becoming Leader. State: Replica: 01643318a337467aa76fa1d896482902, State: Running, Role: LEADER
I20260810 18:57:53.377568 31267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01643318a337467aa76fa1d896482902" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42213 } }
I20260810 18:57:53.378146 31250 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:53.379447 31268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01643318a337467aa76fa1d896482902" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01643318a337467aa76fa1d896482902" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42213 } } }
I20260810 18:57:53.379477 31269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01643318a337467aa76fa1d896482902. Latest consensus state: current_term: 1 leader_uuid: "01643318a337467aa76fa1d896482902" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01643318a337467aa76fa1d896482902" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42213 } } }
I20260810 18:57:53.379603 31268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:53.379609 31269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902 [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:53.380033 31282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:53.383188 31282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:53.389477 31282 catalog_manager.cc:1347] Generated new cluster ID: c86a481a997b4dda977bb2037cab23bd
I20260810 18:57:53.389570 31282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:53.396750 31282 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:53.397641 31282 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:53.406231 31282 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01643318a337467aa76fa1d896482902: Generated new TSK 0
I20260810 18:57:53.407003 31282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:53.412062 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31138
W20260810 18:57:53.426928 31116 connection.cc:535] client connection to 127.0.0.1:42213 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42213): Connection reset by peer (error 104)
2026-08-10T18:57:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1680 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:53 dist-test-slave-68xk krb5kdc[31305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:53 dist-test-slave-68xk krb5kdc[31305](info): set up 2 sockets
Aug 10 18:57:53 dist-test-slave-68xk krb5kdc[31305](info): commencing operation
krb5kdc: starting...
W20260810 18:57:54.724524 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:54 dist-test-slave-68xk krb5kdc[31305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:54.865259 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40327
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:54.978406 31342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:54.978693 31342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:54.978735 31342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:54.982741 31342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:54.982805 31342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:54.982824 31342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:54.982841 31342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:54.982856 31342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:54.988106 31342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:54.989164 31342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:54.990263 31342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:57:54.998839 31342 server_base.cc:1034] running on GCE node
W20260810 18:57:54.998760 31348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:54.998804 31350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:54.998780 31352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:54.999508 31342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:55.000607 31342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:55.001775 31342 hybrid_clock.cc:648] HybridClock initialized: now 1786388275001752 us; error 37 us; skew 500 ppm
Aug 10 18:57:55 dist-test-slave-68xk krb5kdc[31305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:55.004703 31342 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:55.006070 31342 webserver.cc:460] Webserver started at http://127.0.0.1:38129/ using document root <none> and password file <none>
I20260810 18:57:55.006935 31342 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:55.007025 31342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:55.007306 31342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:55.010097 31342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "4a1068d86e5243ef89794878643efb3f"
format_stamp: "Formatted at 2026-08-10 18:57:55 on dist-test-slave-68xk"
server_key: "fdf6b2a65e8e5f804759659d2f66a1ef"
server_key_iv: "0626dcf199ff1ca74cd2b96529455d32"
server_key_version: "encryptionkey@0"
I20260810 18:57:55.010975 31342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "4a1068d86e5243ef89794878643efb3f"
format_stamp: "Formatted at 2026-08-10 18:57:55 on dist-test-slave-68xk"
server_key: "fdf6b2a65e8e5f804759659d2f66a1ef"
server_key_iv: "0626dcf199ff1ca74cd2b96529455d32"
server_key_version: "encryptionkey@0"
I20260810 18:57:55.015529 31342 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:55.017886 31364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:55.019179 31342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:57:55.019340 31342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "4a1068d86e5243ef89794878643efb3f"
format_stamp: "Formatted at 2026-08-10 18:57:55 on dist-test-slave-68xk"
server_key: "fdf6b2a65e8e5f804759659d2f66a1ef"
server_key_iv: "0626dcf199ff1ca74cd2b96529455d32"
server_key_version: "encryptionkey@0"
I20260810 18:57:55.019450 31342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:55.024037 31342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:55.027285 31342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:55.027480 31342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:55.034533 31342 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40327
I20260810 18:57:55.034569 31442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40327 every 8 connection(s)
I20260810 18:57:55.035424 31342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:55.038118 31447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:55.042382 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31342
I20260810 18:57:55.042564 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:55.042870 31334 external_mini_cluster.cc:1428] Setting key d7dc988c74a475aa6d734fb7054c8bc5
I20260810 18:57:55.043993 31447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f: Bootstrap starting.
I20260810 18:57:55.046777 31447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:55.047719 31447 log.cc:824] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:55.048424 31453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:55.044233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:47005 (local address 127.0.0.1:40327)
0810 18:57:55.044664 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:55.044682 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:55.044733 (+ 51us) server_negotiation.cc:407] Connection header received
0810 18:57:55.045587 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:55.045610 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:55.045954 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:55.046304 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:55.047074 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:55.047081 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:55.047098 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:55.047114 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:55.047173 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:55.047198 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:55.047309 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:55.047396 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 18:57:55.047549 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260810 18:57:55.049753 31447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f: No bootstrap required, opened a new log
I20260810 18:57:55.051237 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:55.053876 31447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1068d86e5243ef89794878643efb3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40327 } }
I20260810 18:57:55.054111 31447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:55.054208 31447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a1068d86e5243ef89794878643efb3f, State: Initialized, Role: FOLLOWER
I20260810 18:57:55.054706 31447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1068d86e5243ef89794878643efb3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40327 } }
I20260810 18:57:55.054867 31453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:55.053214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:57057 (local address 127.0.0.1:40327)
0810 18:57:55.053348 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:55.053353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:55.053587 (+ 234us) server_negotiation.cc:407] Connection header received
0810 18:57:55.053650 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:55.053652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:55.053683 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:55.053774 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:55.054488 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:55.054492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:55.054495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:55.054507 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:55.054538 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:55.054551 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:55.054636 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:55.054716 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 18:57:55.054756 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 18:57:55.054908 31447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:55.054973 31447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:55.055078 31447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:55.056154 31447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1068d86e5243ef89794878643efb3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40327 } }
I20260810 18:57:55.056496 31447 leader_election.cc:302] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a1068d86e5243ef89794878643efb3f; no voters:
I20260810 18:57:55.056820 31447 leader_election.cc:288] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:55.056879 31465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:55.057128 31465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [term 1 LEADER]: Becoming Leader. State: Replica: 4a1068d86e5243ef89794878643efb3f, State: Running, Role: LEADER
I20260810 18:57:55.057430 31465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1068d86e5243ef89794878643efb3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40327 } }
I20260810 18:57:55.058123 31447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:55.059513 31468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a1068d86e5243ef89794878643efb3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1068d86e5243ef89794878643efb3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40327 } } }
I20260810 18:57:55.059547 31472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a1068d86e5243ef89794878643efb3f. Latest consensus state: current_term: 1 leader_uuid: "4a1068d86e5243ef89794878643efb3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a1068d86e5243ef89794878643efb3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40327 } } }
I20260810 18:57:55.059645 31468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:55.059652 31472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:55.061244 31483 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:55.063884 31483 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:55.069022 31483 catalog_manager.cc:1347] Generated new cluster ID: 52d0e89f661c49179732485431e4df04
I20260810 18:57:55.069089 31483 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:55.102694 31483 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:55.103076 31483 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:55.112293 31483 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a1068d86e5243ef89794878643efb3f: Generated new TSK 0
I20260810 18:57:55.113065 31483 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:55.124092 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31342
2026-08-10T18:57:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1710 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:55 dist-test-slave-68xk krb5kdc[31501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:55 dist-test-slave-68xk krb5kdc[31501](info): set up 2 sockets
Aug 10 18:57:55 dist-test-slave-68xk krb5kdc[31501](info): commencing operation
krb5kdc: starting...
W20260810 18:57:56.372200 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:56 dist-test-slave-68xk krb5kdc[31501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:56.506505 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36791
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:56.622057 31517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:56.622366 31517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:56.622473 31517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:56.626555 31517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:56.626626 31517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:56.626645 31517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:56.626662 31517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:56.626678 31517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:56.632107 31517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36791
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:56.633221 31517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:56.634230 31517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:57:56.643209 31517 server_base.cc:1034] running on GCE node
W20260810 18:57:56.643297 31526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:56.643150 31530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:56.643150 31528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:56.644124 31517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:56.645251 31517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:56.646461 31517 hybrid_clock.cc:648] HybridClock initialized: now 1786388276646441 us; error 34 us; skew 500 ppm
Aug 10 18:57:56 dist-test-slave-68xk krb5kdc[31501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388276, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:56.653331 31517 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:56.654402 31517 webserver.cc:460] Webserver started at http://127.0.0.1:44803/ using document root <none> and password file <none>
I20260810 18:57:56.655112 31517 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:56.655234 31517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:56.655514 31517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:56.657912 31517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "8a83c60237934b4a8f675049fb672c3c"
format_stamp: "Formatted at 2026-08-10 18:57:56 on dist-test-slave-68xk"
server_key: "0ba71b6c71c0edbd285c8afd0d9b65e0"
server_key_iv: "241bd951fcd20faa878ab73fbc2d6c56"
server_key_version: "encryptionkey@0"
I20260810 18:57:56.658628 31517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "8a83c60237934b4a8f675049fb672c3c"
format_stamp: "Formatted at 2026-08-10 18:57:56 on dist-test-slave-68xk"
server_key: "0ba71b6c71c0edbd285c8afd0d9b65e0"
server_key_iv: "241bd951fcd20faa878ab73fbc2d6c56"
server_key_version: "encryptionkey@0"
I20260810 18:57:56.662982 31517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:56.665817 31538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:56.667450 31517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:56.667624 31517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "8a83c60237934b4a8f675049fb672c3c"
format_stamp: "Formatted at 2026-08-10 18:57:56 on dist-test-slave-68xk"
server_key: "0ba71b6c71c0edbd285c8afd0d9b65e0"
server_key_iv: "241bd951fcd20faa878ab73fbc2d6c56"
server_key_version: "encryptionkey@0"
I20260810 18:57:56.667793 31517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:56.680763 31517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:56.684274 31517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:56.684506 31517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:56.693032 31517 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36791
I20260810 18:57:56.693063 31617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36791 every 8 connection(s)
I20260810 18:57:56.694061 31517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:56.697157 31619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:56.703118 31619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c: Bootstrap starting.
I20260810 18:57:56.704339 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31517
I20260810 18:57:56.704516 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:56.704840 31334 external_mini_cluster.cc:1428] Setting key 218d31465beac7970276a0d727b14fca
I20260810 18:57:56.706274 31619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:56.707118 31619 log.cc:824] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:56.709208 31619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c: No bootstrap required, opened a new log
I20260810 18:57:56.712845 31619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a83c60237934b4a8f675049fb672c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36791 } }
I20260810 18:57:56.713105 31619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:56.713192 31619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a83c60237934b4a8f675049fb672c3c, State: Initialized, Role: FOLLOWER
I20260810 18:57:56.713737 31619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a83c60237934b4a8f675049fb672c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36791 } }
I20260810 18:57:56.713908 31619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Aug 10 18:57:56 dist-test-slave-68xk krb5kdc[31501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 18:57:56.714003 31619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:56.714131 31619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:56.715554 31619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a83c60237934b4a8f675049fb672c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36791 } }
I20260810 18:57:56.716005 31619 leader_election.cc:302] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a83c60237934b4a8f675049fb672c3c; no voters:
I20260810 18:57:56.716454 31619 leader_election.cc:288] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:56.716566 31630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:56.716863 31630 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [term 1 LEADER]: Becoming Leader. State: Replica: 8a83c60237934b4a8f675049fb672c3c, State: Running, Role: LEADER
I20260810 18:57:56.717268 31630 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a83c60237934b4a8f675049fb672c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36791 } }
I20260810 18:57:56.717945 31619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:56.720703 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:56.706076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:52331 (local address 127.0.0.1:36791)
0810 18:57:56.706597 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:56.706613 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:56.706650 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:57:56.707372 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:56.707391 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:56.707829 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:56.708233 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:56.709049 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:56.710074 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:56.711049 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:56.711556 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:56.711727 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:56.714773 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:56.714779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:56.714798 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:56.714835 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:56.717092 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:56.717748 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:56.717755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:56.717761 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:56.717909 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:56.718342 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:56.718345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:56.718347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:56.718921 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:56.719197 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:56.719474 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 18:57:56.719721 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":142,"threads_started":1}
I20260810 18:57:56.721071 31632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a83c60237934b4a8f675049fb672c3c. Latest consensus state: current_term: 1 leader_uuid: "8a83c60237934b4a8f675049fb672c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a83c60237934b4a8f675049fb672c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36791 } } }
I20260810 18:57:56.721454 31632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:56.721288 31633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a83c60237934b4a8f675049fb672c3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a83c60237934b4a8f675049fb672c3c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36791 } } }
I20260810 18:57:56.721684 31633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:56.724525 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 18:57:56.724779 31657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:57:56.724869 31657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:57:56.725961 31659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:56.727023 31659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:56.733623 31659 catalog_manager.cc:1347] Generated new cluster ID: d2a8dbb4ec9e41109f86e01e9c98fc33
I20260810 18:57:56.733695 31659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:56.738453 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:56.726732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:53541 (local address 127.0.0.1:36791)
0810 18:57:56.726862 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:56.726867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:56.729374 (+ 2507us) server_negotiation.cc:407] Connection header received
0810 18:57:56.729492 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:56.729495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:56.729547 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:56.729642 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:56.730884 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:56.731522 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:56.732355 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:56.732622 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:56.732738 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:56.734594 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:56.734598 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:56.734601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:57:56.734629 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:56.736439 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:56.737017 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:56.737021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:56.737023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:56.737128 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:57:56.737480 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:57:56.737483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:57:56.737485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:57:56.737713 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:56.737883 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:56.738134 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 18:57:56.738243 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:57:56.744046 31659 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:56.745148 31659 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:56.749853 31659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a83c60237934b4a8f675049fb672c3c: Generated new TSK 0
I20260810 18:57:56.750725 31659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:56.760253 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31517
W20260810 18:57:56.778366 31499 connection.cc:535] client connection to 127.0.0.1:36791 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36791): Connection reset by peer (error 104)
2026-08-10T18:57:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:56 dist-test-slave-68xk krb5kdc[31672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:56 dist-test-slave-68xk krb5kdc[31672](info): set up 2 sockets
Aug 10 18:57:56 dist-test-slave-68xk krb5kdc[31672](info): commencing operation
krb5kdc: starting...
W20260810 18:57:58.063009 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:58 dist-test-slave-68xk krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:58.202589 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:57:58.318327 31690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:57:58.318636 31690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:57:58.318717 31690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:57:58.322753 31690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:57:58.322817 31690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:57:58.322836 31690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:57:58.322853 31690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:57:58.322868 31690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:57:58.328225 31690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37617
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:57:58.329344 31690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:57:58.330463 31690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:57:58.339653 31697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:58.339659 31696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:57:58.339653 31701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:57:58.339778 31690 server_base.cc:1034] running on GCE node
I20260810 18:57:58.340608 31690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:57:58.341872 31690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:57:58.343086 31690 hybrid_clock.cc:648] HybridClock initialized: now 1786388278343061 us; error 42 us; skew 500 ppm
Aug 10 18:57:58 dist-test-slave-68xk krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388278, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:57:58.346277 31690 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:57:58.347298 31690 webserver.cc:460] Webserver started at http://127.0.0.1:35601/ using document root <none> and password file <none>
I20260810 18:57:58.347898 31690 fs_manager.cc:359] Metadata directory not provided
I20260810 18:57:58.347993 31690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:57:58.348230 31690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:57:58.350383 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "a809b8bc4681435f9338a2b4378e6e8d"
format_stamp: "Formatted at 2026-08-10 18:57:58 on dist-test-slave-68xk"
server_key: "18e293918276b241a17b4efe3882f1f9"
server_key_iv: "3c8d63b2e9f7f482f035ce5147057b02"
server_key_version: "encryptionkey@0"
I20260810 18:57:58.351025 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "a809b8bc4681435f9338a2b4378e6e8d"
format_stamp: "Formatted at 2026-08-10 18:57:58 on dist-test-slave-68xk"
server_key: "18e293918276b241a17b4efe3882f1f9"
server_key_iv: "3c8d63b2e9f7f482f035ce5147057b02"
server_key_version: "encryptionkey@0"
I20260810 18:57:58.354980 31690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:57:58.357857 31709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:57:58.359333 31690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:57:58.359515 31690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "a809b8bc4681435f9338a2b4378e6e8d"
format_stamp: "Formatted at 2026-08-10 18:57:58 on dist-test-slave-68xk"
server_key: "18e293918276b241a17b4efe3882f1f9"
server_key_iv: "3c8d63b2e9f7f482f035ce5147057b02"
server_key_version: "encryptionkey@0"
I20260810 18:57:58.359681 31690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:57:58.392838 31690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:57:58.396531 31690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:57:58.396792 31690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:57:58.405169 31690 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37617
I20260810 18:57:58.405200 31795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37617 every 8 connection(s)
I20260810 18:57:58.406250 31690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:57:58.409116 31796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:57:58.410712 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31690
I20260810 18:57:58.410873 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:57:58.411170 31334 external_mini_cluster.cc:1428] Setting key 32c8b9bba85c986b8b5164d412a8dbd3
I20260810 18:57:58.415689 31796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d: Bootstrap starting.
I20260810 18:57:58.418021 31796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d: Neither blocks nor log segments found. Creating new log.
I20260810 18:57:58.418841 31796 log.cc:824] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d: Log is configured to *not* fsync() on all Append() calls
I20260810 18:57:58.420917 31796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d: No bootstrap required, opened a new log
I20260810 18:57:58.421299 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:58.412780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:48861 (local address 127.0.0.1:37617)
0810 18:57:58.413246 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:58.413262 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:58.413312 (+ 50us) server_negotiation.cc:407] Connection header received
0810 18:57:58.414047 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:58.414072 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:58.414530 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:58.414915 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:58.415376 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:58.416515 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:58.417550 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:58.418111 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:58.418335 (+ 224us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:58.419395 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:58.419401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:58.419416 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:58.419842 (+ 426us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:58.419904 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:58.419928 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:58.420096 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:58.420302 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 18:57:58.420454 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":129,"threads_started":1}
I20260810 18:57:58.424713 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:57:58.425302 31796 raft_consensus.cc:348] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a809b8bc4681435f9338a2b4378e6e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37617 } }
I20260810 18:57:58.425581 31796 raft_consensus.cc:374] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:57:58.425664 31796 raft_consensus.cc:729] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a809b8bc4681435f9338a2b4378e6e8d, State: Initialized, Role: FOLLOWER
I20260810 18:57:58.426189 31796 consensus_queue.cc:260] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a809b8bc4681435f9338a2b4378e6e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37617 } }
I20260810 18:57:58.426379 31796 raft_consensus.cc:388] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:57:58.426479 31796 raft_consensus.cc:482] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:57:58.426587 31796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:57:58.427839 31796 raft_consensus.cc:504] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a809b8bc4681435f9338a2b4378e6e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37617 } }
I20260810 18:57:58.428264 31796 leader_election.cc:302] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a809b8bc4681435f9338a2b4378e6e8d; no voters:
I20260810 18:57:58.428644 31796 leader_election.cc:288] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:57:58.428716 31809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:57:58.428946 31809 raft_consensus.cc:686] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [term 1 LEADER]: Becoming Leader. State: Replica: a809b8bc4681435f9338a2b4378e6e8d, State: Running, Role: LEADER
I20260810 18:57:58.429332 31809 consensus_queue.cc:237] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a809b8bc4681435f9338a2b4378e6e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37617 } }
I20260810 18:57:58.430076 31796 sys_catalog.cc:564] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:57:58.431082 31810 sys_catalog.cc:455] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a809b8bc4681435f9338a2b4378e6e8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a809b8bc4681435f9338a2b4378e6e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37617 } } }
I20260810 18:57:58.431231 31810 sys_catalog.cc:458] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:58.431485 31810 sys_catalog.cc:455] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader a809b8bc4681435f9338a2b4378e6e8d. Latest consensus state: current_term: 1 leader_uuid: "a809b8bc4681435f9338a2b4378e6e8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a809b8bc4681435f9338a2b4378e6e8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37617 } } }
I20260810 18:57:58.431560 31810 sys_catalog.cc:458] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d [sys.catalog]: This master's current role is: LEADER
I20260810 18:57:58.432037 31829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:57:58.432658 31800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:57:58.427313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:41229 (local address 127.0.0.1:37617)
0810 18:57:58.427472 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 18:57:58.427478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:57:58.427676 (+ 198us) server_negotiation.cc:407] Connection header received
0810 18:57:58.427810 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:57:58.427812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:57:58.427859 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:57:58.427934 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:57:58.428268 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:58.429102 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:58.430175 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:57:58.430590 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:57:58.430698 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:57:58.431882 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:57:58.431886 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:57:58.431889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:57:58.431906 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:57:58.431939 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:57:58.431955 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:57:58.432064 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 18:57:58.432454 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 18:57:58.432497 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:57:58.435689 31829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:57:58.441645 31829 catalog_manager.cc:1347] Generated new cluster ID: 789f3ebbf7c64c76a57b792a8e3e306b
I20260810 18:57:58.441728 31829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:57:58.456754 31829 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:57:58.457922 31829 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:57:58.474035 31829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a809b8bc4681435f9338a2b4378e6e8d: Generated new TSK 0
I20260810 18:57:58.474928 31829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:57:58.480903 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31690
W20260810 18:57:58.494318 31670 connection.cc:535] client connection to 127.0.0.1:37617 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37617): Connection reset by peer (error 104)
2026-08-10T18:57:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1716 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:57:58 dist-test-slave-68xk krb5kdc[31856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:57:58 dist-test-slave-68xk krb5kdc[31856](info): set up 2 sockets
Aug 10 18:57:58 dist-test-slave-68xk krb5kdc[31856](info): commencing operation
krb5kdc: starting...
W20260810 18:57:59.775594 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:57:59 dist-test-slave-68xk krb5kdc[31856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:57:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:57:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 18:57:59.914105 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:00.048489 31873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:00.048782 31873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:00.048856 31873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:00.053119 31873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:00.053186 31873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:00.053206 31873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:00.053223 31873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:00.053239 31873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:00.058830 31873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:00.059932 31873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:00.060966 31873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:00.069766 31883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:00.069748 31880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:00.069912 31873 server_base.cc:1034] running on GCE node
W20260810 18:58:00.069741 31886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:00.070715 31873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:00.072352 31873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:00.073530 31873 hybrid_clock.cc:648] HybridClock initialized: now 1786388280073500 us; error 47 us; skew 500 ppm
Aug 10 18:58:00 dist-test-slave-68xk krb5kdc[31856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388280, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:00.076537 31873 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 18:58:00.077450 31873 webserver.cc:460] Webserver started at http://127.0.0.1:37753/ using document root <none> and password file <none>
I20260810 18:58:00.078037 31873 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:00.078135 31873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:00.078387 31873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:00.080554 31873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "91e44747e978477c981187441265d18b"
format_stamp: "Formatted at 2026-08-10 18:58:00 on dist-test-slave-68xk"
server_key: "6dc9ebca0b02d9ed8e167f2c0b4b1b90"
server_key_iv: "867112c2b6cba75623dc6a49f76a1dfb"
server_key_version: "encryptionkey@0"
I20260810 18:58:00.081149 31873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "91e44747e978477c981187441265d18b"
format_stamp: "Formatted at 2026-08-10 18:58:00 on dist-test-slave-68xk"
server_key: "6dc9ebca0b02d9ed8e167f2c0b4b1b90"
server_key_iv: "867112c2b6cba75623dc6a49f76a1dfb"
server_key_version: "encryptionkey@0"
I20260810 18:58:00.085302 31873 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:58:00.087965 31895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:00.089180 31873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 18:58:00.089378 31873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "91e44747e978477c981187441265d18b"
format_stamp: "Formatted at 2026-08-10 18:58:00 on dist-test-slave-68xk"
server_key: "6dc9ebca0b02d9ed8e167f2c0b4b1b90"
server_key_iv: "867112c2b6cba75623dc6a49f76a1dfb"
server_key_version: "encryptionkey@0"
I20260810 18:58:00.089500 31873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:00.094340 31873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:00.097754 31873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:00.097955 31873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:00.105432 31873 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37675
I20260810 18:58:00.105742 31973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37675 every 8 connection(s)
I20260810 18:58:00.106537 31873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:00.109515 31975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:00.112495 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 31873
I20260810 18:58:00.112648 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:00.112950 31334 external_mini_cluster.cc:1428] Setting key 47e3c1e02128f3c7a43c5506216131ba
I20260810 18:58:00.116079 31975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b: Bootstrap starting.
I20260810 18:58:00.118969 31975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:00.118974 31979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:00.114580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:57009 (local address 127.0.0.1:37675)
0810 18:58:00.115074 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:00.115095 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:00.115141 (+ 46us) server_negotiation.cc:407] Connection header received
0810 18:58:00.115990 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:00.116014 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:00.116381 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:00.116751 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:00.117516 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:00.117524 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:00.117540 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:00.117553 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:00.117627 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:00.117655 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:00.117797 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:00.117864 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 18:58:00.118017 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":132,"threads_started":1}
I20260810 18:58:00.119827 31975 log.cc:824] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:00.121822 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:58:00.122043 31975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b: No bootstrap required, opened a new log
I20260810 18:58:00.125881 31979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:00.124040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:52789 (local address 127.0.0.1:37675)
0810 18:58:00.124185 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:00.124189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:00.124415 (+ 226us) server_negotiation.cc:407] Connection header received
0810 18:58:00.124500 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:00.124502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:00.124534 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:00.124616 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:00.125429 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:00.125432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:00.125434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:00.125444 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:00.125468 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:00.125478 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:00.125550 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:00.125682 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 18:58:00.125719 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:58:00.126619 31975 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e44747e978477c981187441265d18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } }
I20260810 18:58:00.126893 31975 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:58:00.126960 31975 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e44747e978477c981187441265d18b, State: Initialized, Role: FOLLOWER
I20260810 18:58:00.127362 31975 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e44747e978477c981187441265d18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } }
I20260810 18:58:00.127493 31975 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:58:00.127540 31975 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:58:00.127614 31975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:00.128753 31975 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e44747e978477c981187441265d18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } }
I20260810 18:58:00.129158 31975 leader_election.cc:302] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91e44747e978477c981187441265d18b; no voters:
I20260810 18:58:00.129503 31975 leader_election.cc:288] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:58:00.129585 31992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:00.129876 31992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [term 1 LEADER]: Becoming Leader. State: Replica: 91e44747e978477c981187441265d18b, State: Running, Role: LEADER
I20260810 18:58:00.130208 31992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e44747e978477c981187441265d18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } }
I20260810 18:58:00.130821 31975 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:00.132115 31995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91e44747e978477c981187441265d18b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e44747e978477c981187441265d18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } } }
I20260810 18:58:00.132272 31995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:00.132149 31998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91e44747e978477c981187441265d18b. Latest consensus state: current_term: 1 leader_uuid: "91e44747e978477c981187441265d18b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e44747e978477c981187441265d18b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37675 } } }
I20260810 18:58:00.132522 31998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b [sys.catalog]: This master's current role is: LEADER
W20260810 18:58:00.136475 32032 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:00.136544 32032 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:00.136624 32019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:00.137331 32019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:00.142386 32019 catalog_manager.cc:1347] Generated new cluster ID: ca3dda3a09a34ba58e6cff5d53187add
I20260810 18:58:00.142478 32019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:00.157541 32019 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:00.157922 32019 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:00.163489 32019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91e44747e978477c981187441265d18b: Generated new TSK 0
I20260810 18:58:00.164176 32019 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:58:00.171594 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 31873
2026-08-10T18:58:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1689 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:58:00 dist-test-slave-68xk krb5kdc[32041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:58:00 dist-test-slave-68xk krb5kdc[32041](info): set up 2 sockets
Aug 10 18:58:00 dist-test-slave-68xk krb5kdc[32041](info): commencing operation
krb5kdc: starting...
W20260810 18:58:01.479993 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:58:01 dist-test-slave-68xk krb5kdc[32041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:58:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:58:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:58:01.613689 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:36807
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:43671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:36807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:01.730798 32058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:01.731103 32058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:01.731205 32058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:01.735245 32058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:01.735337 32058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:01.735373 32058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:01.735433 32058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:01.735491 32058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:01.740938 32058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:43671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:36807
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:36807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:01.742115 32058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:01.743259 32058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:01.752213 32070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:01.752262 32072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:01.752271 32067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:01.752411 32058 server_base.cc:1034] running on GCE node
I20260810 18:58:01.753176 32058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:01.754274 32058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:01.755452 32058 hybrid_clock.cc:648] HybridClock initialized: now 1786388281755434 us; error 32 us; skew 500 ppm
Aug 10 18:58:01 dist-test-slave-68xk krb5kdc[32041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388281, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:01.763707 32058 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:58:01.764799 32058 webserver.cc:460] Webserver started at http://127.30.153.190:35151/ using document root <none> and password file <none>
I20260810 18:58:01.765455 32058 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:01.765555 32058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:01.765817 32058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:01.768097 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "5e41d18b6e5443129d35800ef6edc803"
format_stamp: "Formatted at 2026-08-10 18:58:01 on dist-test-slave-68xk"
server_key: "70dc0ffdf1ef78b3bf04c309b4c25f72"
server_key_iv: "aa71d0a92759812b7d68ca20e8af782a"
server_key_version: "encryptionkey@0"
I20260810 18:58:01.768719 32058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "5e41d18b6e5443129d35800ef6edc803"
format_stamp: "Formatted at 2026-08-10 18:58:01 on dist-test-slave-68xk"
server_key: "70dc0ffdf1ef78b3bf04c309b4c25f72"
server_key_iv: "aa71d0a92759812b7d68ca20e8af782a"
server_key_version: "encryptionkey@0"
I20260810 18:58:01.772737 32058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 18:58:01.775708 32082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:01.777572 32058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:58:01.777750 32058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "5e41d18b6e5443129d35800ef6edc803"
format_stamp: "Formatted at 2026-08-10 18:58:01 on dist-test-slave-68xk"
server_key: "70dc0ffdf1ef78b3bf04c309b4c25f72"
server_key_iv: "aa71d0a92759812b7d68ca20e8af782a"
server_key_version: "encryptionkey@0"
I20260810 18:58:01.777899 32058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:01.800263 32058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:01.803766 32058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:01.804003 32058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:01.812577 32058 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:36807
I20260810 18:58:01.812654 32158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:36807 every 8 connection(s)
I20260810 18:58:01.813670 32058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:01.816627 32160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:01.821516 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 32058
I20260810 18:58:01.821684 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:01.821952 31334 external_mini_cluster.cc:1428] Setting key 5af625d7dbc55299952ee9239ee87558
I20260810 18:58:01.822365 32160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803: Bootstrap starting.
I20260810 18:58:01.825135 32160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:01.826046 32160 log.cc:824] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:01.828438 32160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803: No bootstrap required, opened a new log
Aug 10 18:58:01 dist-test-slave-68xk krb5kdc[32041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:58:01.832748 32160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e41d18b6e5443129d35800ef6edc803" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36807 } }
I20260810 18:58:01.833047 32160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:58:01.833120 32160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e41d18b6e5443129d35800ef6edc803, State: Initialized, Role: FOLLOWER
I20260810 18:58:01.833683 32160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e41d18b6e5443129d35800ef6edc803" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36807 } }
I20260810 18:58:01.833868 32160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:58:01.833941 32160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:58:01.834045 32160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:01.835286 32160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e41d18b6e5443129d35800ef6edc803" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36807 } }
I20260810 18:58:01.835664 32160 leader_election.cc:302] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e41d18b6e5443129d35800ef6edc803; no voters:
I20260810 18:58:01.836059 32160 leader_election.cc:288] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:58:01.836216 32169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:01.836499 32169 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [term 1 LEADER]: Becoming Leader. State: Replica: 5e41d18b6e5443129d35800ef6edc803, State: Running, Role: LEADER
I20260810 18:58:01.836876 32169 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e41d18b6e5443129d35800ef6edc803" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36807 } }
I20260810 18:58:01.837447 32160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:01.838452 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:01.823470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53797 (local address 127.30.153.190:36807)
0810 18:58:01.823951 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:01.823970 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:01.824015 (+ 45us) server_negotiation.cc:407] Connection header received
0810 18:58:01.824845 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:01.824869 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:01.825342 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:01.825726 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:01.826558 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:01.827992 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:01.828971 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:01.829652 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:01.829846 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:01.832508 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:01.832515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:01.832530 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:01.832562 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:01.834760 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:01.835292 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:01.835300 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:01.835308 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:01.835456 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:01.835813 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:01.835817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:01.835818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:01.836388 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:01.836605 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:01.837136 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 18:58:01.837378 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":140,"threads_started":1}
I20260810 18:58:01.838860 32172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e41d18b6e5443129d35800ef6edc803. Latest consensus state: current_term: 1 leader_uuid: "5e41d18b6e5443129d35800ef6edc803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e41d18b6e5443129d35800ef6edc803" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36807 } } }
I20260810 18:58:01.838994 32172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:01.839334 32170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e41d18b6e5443129d35800ef6edc803" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e41d18b6e5443129d35800ef6edc803" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36807 } } }
I20260810 18:58:01.839435 32170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:01.841893 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 18:58:01.842587 32192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:01.842672 32192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:01.842746 32185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:01.843572 32185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:01.850000 32185 catalog_manager.cc:1347] Generated new cluster ID: 4dbf422a5e1b43d4ba248f2c4b694b67
I20260810 18:58:01.850061 32185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:01.853605 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:01.844300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40723 (local address 127.30.153.190:36807)
0810 18:58:01.844463 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:01.844468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:01.844711 (+ 243us) server_negotiation.cc:407] Connection header received
0810 18:58:01.844862 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:01.844866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:01.844913 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:01.845005 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:01.845853 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:01.846694 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:01.847853 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:01.848122 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:01.848218 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:01.849982 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:01.849986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:01.849988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:01.850016 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:01.851743 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:01.852308 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:01.852311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:01.852313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:01.852391 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:01.852741 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:01.852744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:01.852746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:01.852925 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:01.853049 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:01.853346 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 18:58:01.853434 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 18:58:01.860826 32185 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:01.861678 32185 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:01.872394 32185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e41d18b6e5443129d35800ef6edc803: Generated new TSK 0
I20260810 18:58:01.873111 32185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:58:01.878233 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 32058
W20260810 18:58:01.893492 32039 connection.cc:535] client connection to 127.30.153.190:36807 recv error: Network error: failed to read from TLS socket (remote: 127.30.153.190:36807): Connection reset by peer (error 104)
2026-08-10T18:58:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1709 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:58:01 dist-test-slave-68xk krb5kdc[32209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:58:01 dist-test-slave-68xk krb5kdc[32209](info): set up 2 sockets
Aug 10 18:58:01 dist-test-slave-68xk krb5kdc[32209](info): commencing operation
krb5kdc: starting...
W20260810 18:58:03.139544 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:58:03 dist-test-slave-68xk krb5kdc[32209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:58:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:58:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:58:03.269387 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:44771
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:44771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:03.384253 32226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:03.384557 32226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:03.384652 32226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:03.388782 32226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:03.388863 32226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:03.388899 32226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:03.388957 32226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:03.388993 32226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:03.394186 32226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:44771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:44771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:03.395282 32226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:03.396392 32226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:03.405084 32238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:03.405098 32234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:03.405102 32235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:03.405169 32226 server_base.cc:1034] running on GCE node
I20260810 18:58:03.405982 32226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:03.407128 32226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:03.408314 32226 hybrid_clock.cc:648] HybridClock initialized: now 1786388283408286 us; error 42 us; skew 500 ppm
Aug 10 18:58:03 dist-test-slave-68xk krb5kdc[32209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388283, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:03.411015 32226 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:58:03.411960 32226 webserver.cc:460] Webserver started at http://127.30.153.190:41681/ using document root <none> and password file <none>
I20260810 18:58:03.412534 32226 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:03.412629 32226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:03.412864 32226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:03.414995 32226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "dad91e77e5d74a16a5a9e57783f53365"
format_stamp: "Formatted at 2026-08-10 18:58:03 on dist-test-slave-68xk"
server_key: "6402f52fa09db11bacf6bda4eb390df8"
server_key_iv: "9714fdd00496c361c255cdc3e8c5d745"
server_key_version: "encryptionkey@0"
I20260810 18:58:03.415580 32226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "dad91e77e5d74a16a5a9e57783f53365"
format_stamp: "Formatted at 2026-08-10 18:58:03 on dist-test-slave-68xk"
server_key: "6402f52fa09db11bacf6bda4eb390df8"
server_key_iv: "9714fdd00496c361c255cdc3e8c5d745"
server_key_version: "encryptionkey@0"
I20260810 18:58:03.419366 32226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 18:58:03.421717 32254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:03.423020 32226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:58:03.423207 32226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "dad91e77e5d74a16a5a9e57783f53365"
format_stamp: "Formatted at 2026-08-10 18:58:03 on dist-test-slave-68xk"
server_key: "6402f52fa09db11bacf6bda4eb390df8"
server_key_iv: "9714fdd00496c361c255cdc3e8c5d745"
server_key_version: "encryptionkey@0"
I20260810 18:58:03.423323 32226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:03.458989 32226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:03.462077 32226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:03.462333 32226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:03.469774 32226 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:44771
I20260810 18:58:03.469802 32326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:44771 every 8 connection(s)
I20260810 18:58:03.470767 32226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:03.473466 32329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:03.477195 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 32226
I20260810 18:58:03.477341 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:03.477629 31334 external_mini_cluster.cc:1428] Setting key 4e28df058ab79b3186dc978ec11327d2
I20260810 18:58:03.479339 32329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365: Bootstrap starting.
I20260810 18:58:03.481643 32329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:03.482477 32329 log.cc:824] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:03.484300 32329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365: No bootstrap required, opened a new log
I20260810 18:58:03.486667 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:03.479026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57943 (local address 127.30.153.190:44771)
0810 18:58:03.479505 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:03.479521 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:03.479556 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:58:03.480183 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:03.480200 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:03.480523 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:03.480923 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:03.481335 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:03.482493 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:03.483408 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:03.483878 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:03.484049 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:03.484969 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:03.484975 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:03.484988 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:03.485377 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:03.485431 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:03.485454 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:03.485598 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:03.485780 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 18:58:03.485911 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":132,"threads_started":1}
I20260810 18:58:03.488050 32329 raft_consensus.cc:348] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad91e77e5d74a16a5a9e57783f53365" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44771 } }
I20260810 18:58:03.488317 32329 raft_consensus.cc:374] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:58:03.488440 32329 raft_consensus.cc:729] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad91e77e5d74a16a5a9e57783f53365, State: Initialized, Role: FOLLOWER
I20260810 18:58:03.488901 32329 consensus_queue.cc:260] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad91e77e5d74a16a5a9e57783f53365" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44771 } }
I20260810 18:58:03.489096 32329 raft_consensus.cc:388] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:58:03.489168 32329 raft_consensus.cc:482] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:58:03.489275 32329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:03.489758 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:58:03.490298 32329 raft_consensus.cc:504] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad91e77e5d74a16a5a9e57783f53365" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44771 } }
I20260810 18:58:03.490638 32329 leader_election.cc:302] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dad91e77e5d74a16a5a9e57783f53365; no voters:
I20260810 18:58:03.490926 32329 leader_election.cc:288] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:58:03.491045 32334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:03.491344 32334 raft_consensus.cc:686] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [term 1 LEADER]: Becoming Leader. State: Replica: dad91e77e5d74a16a5a9e57783f53365, State: Running, Role: LEADER
I20260810 18:58:03.491673 32334 consensus_queue.cc:237] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad91e77e5d74a16a5a9e57783f53365" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44771 } }
I20260810 18:58:03.492219 32329 sys_catalog.cc:564] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:03.492954 32340 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dad91e77e5d74a16a5a9e57783f53365. Latest consensus state: current_term: 1 leader_uuid: "dad91e77e5d74a16a5a9e57783f53365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad91e77e5d74a16a5a9e57783f53365" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44771 } } }
I20260810 18:58:03.493283 32340 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:03.493165 32345 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dad91e77e5d74a16a5a9e57783f53365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad91e77e5d74a16a5a9e57783f53365" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44771 } } }
I20260810 18:58:03.493640 32345 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:03.493955 32356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:03.497047 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:03.492053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58673 (local address 127.30.153.190:44771)
0810 18:58:03.492149 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:03.492153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:03.492442 (+ 289us) server_negotiation.cc:407] Connection header received
0810 18:58:03.492547 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:03.492550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:03.492597 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:03.492687 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:03.493171 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:03.494003 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:03.495185 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:03.495493 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:03.495603 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:03.496543 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:03.496547 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:03.496550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:03.496564 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:03.496592 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:03.496611 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:03.496706 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:03.496872 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 18:58:03.496907 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":38}
I20260810 18:58:03.497776 32356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:03.503943 32356 catalog_manager.cc:1347] Generated new cluster ID: af15ec72c7a54c10bb3504d468db5a48
I20260810 18:58:03.504027 32356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:03.509862 32356 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:03.510919 32356 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:03.517577 32356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dad91e77e5d74a16a5a9e57783f53365: Generated new TSK 0
I20260810 18:58:03.518483 32356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:58:03.529450 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 32226
W20260810 18:58:03.543375 32204 connection.cc:535] client connection to 127.30.153.190:44771 recv error: Network error: failed to read from TLS socket (remote: 127.30.153.190:44771): Connection reset by peer (error 104)
2026-08-10T18:58:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:58:03 dist-test-slave-68xk krb5kdc[32390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:58:03 dist-test-slave-68xk krb5kdc[32390](info): set up 2 sockets
Aug 10 18:58:03 dist-test-slave-68xk krb5kdc[32390](info): commencing operation
krb5kdc: starting...
W20260810 18:58:04.859275 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:58:04 dist-test-slave-68xk krb5kdc[32390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:58:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:58:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:58:04.993322 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:36829
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:45371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:36829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:05.111243 32412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:05.111549 32412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:05.111646 32412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:05.115679 32412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:05.115767 32412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:05.115805 32412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:05.115865 32412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:05.115902 32412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:05.121309 32412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:45371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:36829
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:36829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:05.122385 32412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:05.123463 32412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 18:58:05.132071 32412 server_base.cc:1034] running on GCE node
W20260810 18:58:05.132062 32422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:05.132028 32426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:05.132001 32420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:05.133008 32412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:05.134058 32412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:05.135223 32412 hybrid_clock.cc:648] HybridClock initialized: now 1786388285135183 us; error 56 us; skew 500 ppm
Aug 10 18:58:05 dist-test-slave-68xk krb5kdc[32390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388285, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:05.138087 32412 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:58:05.138995 32412 webserver.cc:460] Webserver started at http://127.30.153.190:35551/ using document root <none> and password file <none>
I20260810 18:58:05.139528 32412 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:05.139590 32412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:05.139751 32412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:05.141825 32412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "cac915d11afa43739bec7d46c978c3b2"
format_stamp: "Formatted at 2026-08-10 18:58:05 on dist-test-slave-68xk"
server_key: "fe0c5cb4d89789b3111d1795c0f05401"
server_key_iv: "b1525cb87f862ba480b8049cc794fce9"
server_key_version: "encryptionkey@0"
I20260810 18:58:05.142391 32412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "cac915d11afa43739bec7d46c978c3b2"
format_stamp: "Formatted at 2026-08-10 18:58:05 on dist-test-slave-68xk"
server_key: "fe0c5cb4d89789b3111d1795c0f05401"
server_key_iv: "b1525cb87f862ba480b8049cc794fce9"
server_key_version: "encryptionkey@0"
I20260810 18:58:05.146697 32412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 18:58:05.149461 32438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:05.150970 32412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:58:05.151139 32412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "cac915d11afa43739bec7d46c978c3b2"
format_stamp: "Formatted at 2026-08-10 18:58:05 on dist-test-slave-68xk"
server_key: "fe0c5cb4d89789b3111d1795c0f05401"
server_key_iv: "b1525cb87f862ba480b8049cc794fce9"
server_key_version: "encryptionkey@0"
I20260810 18:58:05.151259 32412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:05.156586 32412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:05.160131 32412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:05.160362 32412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:05.168210 32412 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:36829
I20260810 18:58:05.168267 32518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:36829 every 8 connection(s)
I20260810 18:58:05.169284 32412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:05.170769 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 32412
I20260810 18:58:05.170926 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:05.171308 31334 external_mini_cluster.cc:1428] Setting key d426769ef2bda3993b373dbfeada7e2b
I20260810 18:58:05.172642 32520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:05.177091 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:05.172697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53607 (local address 127.30.153.190:36829)
0810 18:58:05.173201 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:05.173220 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:05.173273 (+ 53us) server_negotiation.cc:407] Connection header received
0810 18:58:05.174109 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:05.174133 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:05.174532 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:05.174886 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:05.175663 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:05.175669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:05.175684 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:05.175696 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:05.175761 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:05.175789 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:05.175930 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:05.175999 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 18:58:05.176153 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":135,"threads_started":1}
I20260810 18:58:05.179826 32520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2: Bootstrap starting.
I20260810 18:58:05.180295 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:58:05.182188 32520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:05.183218 32520 log.cc:824] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:05.184482 32525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:05.182632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43649 (local address 127.30.153.190:36829)
0810 18:58:05.182789 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:05.182793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:05.183061 (+ 268us) server_negotiation.cc:407] Connection header received
0810 18:58:05.183219 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:05.183222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:05.183258 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:05.183377 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:05.184109 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:05.184113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:05.184116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:05.184128 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:05.184156 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:05.184170 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:05.184277 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:05.184332 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 18:58:05.184370 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:58:05.185719 32520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2: No bootstrap required, opened a new log
I20260810 18:58:05.189296 32520 raft_consensus.cc:348] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac915d11afa43739bec7d46c978c3b2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36829 } }
I20260810 18:58:05.189515 32520 raft_consensus.cc:374] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:58:05.189611 32520 raft_consensus.cc:729] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac915d11afa43739bec7d46c978c3b2, State: Initialized, Role: FOLLOWER
I20260810 18:58:05.190052 32520 consensus_queue.cc:260] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac915d11afa43739bec7d46c978c3b2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36829 } }
I20260810 18:58:05.190197 32520 raft_consensus.cc:388] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:58:05.190281 32520 raft_consensus.cc:482] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:58:05.190402 32520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:05.191478 32520 raft_consensus.cc:504] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac915d11afa43739bec7d46c978c3b2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36829 } }
I20260810 18:58:05.191824 32520 leader_election.cc:302] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cac915d11afa43739bec7d46c978c3b2; no voters:
I20260810 18:58:05.192178 32520 leader_election.cc:288] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:58:05.192260 32539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:05.192543 32539 raft_consensus.cc:686] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [term 1 LEADER]: Becoming Leader. State: Replica: cac915d11afa43739bec7d46c978c3b2, State: Running, Role: LEADER
I20260810 18:58:05.192896 32539 consensus_queue.cc:237] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac915d11afa43739bec7d46c978c3b2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36829 } }
I20260810 18:58:05.193472 32520 sys_catalog.cc:564] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:05.194772 32542 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cac915d11afa43739bec7d46c978c3b2. Latest consensus state: current_term: 1 leader_uuid: "cac915d11afa43739bec7d46c978c3b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac915d11afa43739bec7d46c978c3b2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36829 } } }
I20260810 18:58:05.194777 32540 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cac915d11afa43739bec7d46c978c3b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac915d11afa43739bec7d46c978c3b2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 36829 } } }
I20260810 18:58:05.194916 32542 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:05.194926 32540 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:05.195493 32556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:05.198392 32556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:05.198755 32571 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 18:58:05.204317 32556 catalog_manager.cc:1347] Generated new cluster ID: 8a7d539934184755b56b9aaf33b7a1df
I20260810 18:58:05.204406 32556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:05.223456 32556 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:05.223802 32556 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:05.231743 32556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cac915d11afa43739bec7d46c978c3b2: Generated new TSK 0
I20260810 18:58:05.232543 32556 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:58:05.236827 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 32412
2026-08-10T18:58:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1704 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:58:05 dist-test-slave-68xk krb5kdc[32577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:58:05 dist-test-slave-68xk krb5kdc[32577](info): set up 2 sockets
Aug 10 18:58:05 dist-test-slave-68xk krb5kdc[32577](info): commencing operation
krb5kdc: starting...
W20260810 18:58:06.480724 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:58:06 dist-test-slave-68xk krb5kdc[32577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:58:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:58:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:58:06.613091 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42339
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:42339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:06.727123 32596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:06.727473 32596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:06.727573 32596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:06.731773 32596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:06.731865 32596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:06.731904 32596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:06.731961 32596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:06.731999 32596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:06.737483 32596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:42339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:42339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:06.738732 32596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:06.739842 32596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:06.749011 32602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:06.749069 32606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:06.749037 32596 server_base.cc:1034] running on GCE node
W20260810 18:58:06.749037 32603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:06.749847 32596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:06.751184 32596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:06.752517 32596 hybrid_clock.cc:648] HybridClock initialized: now 1786388286752517 us; error 48 us; skew 500 ppm
Aug 10 18:58:06 dist-test-slave-68xk krb5kdc[32577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388286, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:06.755511 32596 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:58:06.756476 32596 webserver.cc:460] Webserver started at http://127.30.153.190:39387/ using document root <none> and password file <none>
I20260810 18:58:06.757133 32596 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:06.757241 32596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:06.757495 32596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:06.759743 32596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "49dee1671b514bb0a85fae49b94d23c2"
format_stamp: "Formatted at 2026-08-10 18:58:06 on dist-test-slave-68xk"
server_key: "a6c23fc45c34059614573b4ce9604120"
server_key_iv: "6ed45bbc23c34667cc078d855ba3c5f3"
server_key_version: "encryptionkey@0"
I20260810 18:58:06.760372 32596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "49dee1671b514bb0a85fae49b94d23c2"
format_stamp: "Formatted at 2026-08-10 18:58:06 on dist-test-slave-68xk"
server_key: "a6c23fc45c34059614573b4ce9604120"
server_key_iv: "6ed45bbc23c34667cc078d855ba3c5f3"
server_key_version: "encryptionkey@0"
I20260810 18:58:06.764484 32596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:58:06.767239 32616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:06.768668 32596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:58:06.768857 32596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "49dee1671b514bb0a85fae49b94d23c2"
format_stamp: "Formatted at 2026-08-10 18:58:06 on dist-test-slave-68xk"
server_key: "a6c23fc45c34059614573b4ce9604120"
server_key_iv: "6ed45bbc23c34667cc078d855ba3c5f3"
server_key_version: "encryptionkey@0"
I20260810 18:58:06.768996 32596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:06.792862 32596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:06.796186 32596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:06.796419 32596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:06.805277 32596 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:42339
I20260810 18:58:06.805291 32700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:42339 every 8 connection(s)
I20260810 18:58:06.806324 32596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:06.809149 32702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:06.811344 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 32596
I20260810 18:58:06.811497 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:06.811798 31334 external_mini_cluster.cc:1428] Setting key 8ce815ee761e2fbc3e7d1166c34a6b0a
I20260810 18:58:06.815104 32702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2: Bootstrap starting.
I20260810 18:58:06.817301 32702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:06.818038 32702 log.cc:824] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:06.820014 32702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2: No bootstrap required, opened a new log
Aug 10 18:58:06 dist-test-slave-68xk krb5kdc[32577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:58:06.823499 32702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49dee1671b514bb0a85fae49b94d23c2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42339 } }
I20260810 18:58:06.823714 32702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:58:06.823782 32702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49dee1671b514bb0a85fae49b94d23c2, State: Initialized, Role: FOLLOWER
I20260810 18:58:06.824268 32702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49dee1671b514bb0a85fae49b94d23c2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42339 } }
I20260810 18:58:06.824421 32702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:58:06.824494 32702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:58:06.824635 32702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:06.825563 32702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49dee1671b514bb0a85fae49b94d23c2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42339 } }
I20260810 18:58:06.825919 32702 leader_election.cc:302] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49dee1671b514bb0a85fae49b94d23c2; no voters:
I20260810 18:58:06.826260 32702 leader_election.cc:288] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:58:06.826391 32709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:06.826678 32709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [term 1 LEADER]: Becoming Leader. State: Replica: 49dee1671b514bb0a85fae49b94d23c2, State: Running, Role: LEADER
I20260810 18:58:06.827028 32709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49dee1671b514bb0a85fae49b94d23c2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42339 } }
I20260810 18:58:06.827459 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:06.813185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38969 (local address 127.30.153.190:42339)
0810 18:58:06.813657 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:06.813673 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:06.813710 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:58:06.814405 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:06.814486 (+ 81us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:06.814847 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:06.815192 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:06.816053 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:06.817054 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:06.818041 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:06.818636 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:06.818814 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:06.821692 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:06.821698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:06.821717 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:06.821744 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:06.824196 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:06.824716 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:06.824721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:06.824726 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:06.824877 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:06.825201 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:06.825205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:06.825207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:06.825615 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:06.825834 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:06.826143 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 18:58:06.826488 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":138,"threads_started":1}
I20260810 18:58:06.827585 32702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:06.829873 32710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49dee1671b514bb0a85fae49b94d23c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49dee1671b514bb0a85fae49b94d23c2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42339 } } }
I20260810 18:58:06.830024 32710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:06.830118 32709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49dee1671b514bb0a85fae49b94d23c2. Latest consensus state: current_term: 1 leader_uuid: "49dee1671b514bb0a85fae49b94d23c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49dee1671b514bb0a85fae49b94d23c2" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 42339 } } }
I20260810 18:58:06.830816 32709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:06.831266 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260810 18:58:06.833143 32729 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:06.833222 32729 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:06.833441 32730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:06.834446 32730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:06.839957 32730 catalog_manager.cc:1347] Generated new cluster ID: 83476c6be0b34d0e824c7b3bd6c45066
I20260810 18:58:06.840034 32730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:06.843381 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:06.833621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59389 (local address 127.30.153.190:42339)
0810 18:58:06.833719 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:06.833724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:06.834154 (+ 430us) server_negotiation.cc:407] Connection header received
0810 18:58:06.834276 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:06.834279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:06.834329 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:06.834452 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:06.835303 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:06.836068 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:06.837270 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:06.837620 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:06.837732 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:06.839596 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:06.839601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:06.839606 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:06.839640 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:06.841532 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:06.842024 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:06.842028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:06.842030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:06.842130 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:06.842454 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:06.842458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:06.842460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:06.842686 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:06.842820 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:06.843069 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 18:58:06.843216 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260810 18:58:06.856421 32730 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:06.857635 32730 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:06.864619 32730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49dee1671b514bb0a85fae49b94d23c2: Generated new TSK 0
I20260810 18:58:06.865455 32730 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:58:06.869650 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 32596
2026-08-10T18:58:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1632 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:58:06 dist-test-slave-68xk krb5kdc[32754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:58:06 dist-test-slave-68xk krb5kdc[32754](info): set up 2 sockets
Aug 10 18:58:06 dist-test-slave-68xk krb5kdc[32754](info): commencing operation
krb5kdc: starting...
W20260810 18:58:08.112419 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:58:08 dist-test-slave-68xk krb5kdc[32754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:58:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:58:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:58:08.251376 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:34085
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:34085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:08.364965 311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:08.365252 311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:08.365293 311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:08.369436 311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:08.369504 311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:08.369524 311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:08.369540 311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:08.369556 311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:08.374790 311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:34085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:34085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:08.375871 311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:08.376909 311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:08.385635 318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:08.385700 320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:08.385632 326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:08.386010 311 server_base.cc:1034] running on GCE node
I20260810 18:58:08.386737 311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:08.387837 311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:08.389015 311 hybrid_clock.cc:648] HybridClock initialized: now 1786388288388982 us; error 50 us; skew 500 ppm
Aug 10 18:58:08 dist-test-slave-68xk krb5kdc[32754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388288, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:08.392010 311 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:58:08.392932 311 webserver.cc:460] Webserver started at http://127.30.153.190:44191/ using document root <none> and password file <none>
I20260810 18:58:08.393541 311 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:08.393604 311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:08.393829 311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:08.395993 311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "862b61697a984e089623807d56a09ae3"
format_stamp: "Formatted at 2026-08-10 18:58:08 on dist-test-slave-68xk"
server_key: "901d1d7c006b437851065c4c5266c70e"
server_key_iv: "837b8413afc8649384b4b17c5515d2e7"
server_key_version: "encryptionkey@0"
I20260810 18:58:08.396620 311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "862b61697a984e089623807d56a09ae3"
format_stamp: "Formatted at 2026-08-10 18:58:08 on dist-test-slave-68xk"
server_key: "901d1d7c006b437851065c4c5266c70e"
server_key_iv: "837b8413afc8649384b4b17c5515d2e7"
server_key_version: "encryptionkey@0"
I20260810 18:58:08.400487 311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:58:08.403043 337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:08.404330 311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 18:58:08.404479 311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "862b61697a984e089623807d56a09ae3"
format_stamp: "Formatted at 2026-08-10 18:58:08 on dist-test-slave-68xk"
server_key: "901d1d7c006b437851065c4c5266c70e"
server_key_iv: "837b8413afc8649384b4b17c5515d2e7"
server_key_version: "encryptionkey@0"
I20260810 18:58:08.404590 311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:08.437681 311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:08.441514 311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:08.441776 311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:08.449913 311 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:34085
I20260810 18:58:08.449928 412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:34085 every 8 connection(s)
I20260810 18:58:08.451007 311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:08.453907 415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:08.459069 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 311
I20260810 18:58:08.459213 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:08.459499 31334 external_mini_cluster.cc:1428] Setting key ba3737562a4169527b2c7666784ced24
I20260810 18:58:08.459669 415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3: Bootstrap starting.
I20260810 18:58:08.462147 415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:08.463085 415 log.cc:824] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:08.465236 415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3: No bootstrap required, opened a new log
I20260810 18:58:08.468761 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:08.460888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58483 (local address 127.30.153.190:34085)
0810 18:58:08.461314 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:08.461330 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:08.461381 (+ 51us) server_negotiation.cc:407] Connection header received
0810 18:58:08.461989 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:08.462005 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:08.462382 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:08.462719 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:08.463109 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:08.464307 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:08.465517 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:08.465981 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:08.466101 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:08.466954 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:08.466959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:08.466971 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:08.467343 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:08.467393 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:08.467415 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:08.467490 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:08.467708 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 18:58:08.467977 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":134,"threads_started":1}
I20260810 18:58:08.469283 415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b61697a984e089623807d56a09ae3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34085 } }
I20260810 18:58:08.469527 415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:58:08.469609 415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 862b61697a984e089623807d56a09ae3, State: Initialized, Role: FOLLOWER
I20260810 18:58:08.470041 415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b61697a984e089623807d56a09ae3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34085 } }
I20260810 18:58:08.470189 415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:58:08.470279 415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:58:08.470402 415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:08.471498 415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b61697a984e089623807d56a09ae3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34085 } }
I20260810 18:58:08.471875 415 leader_election.cc:302] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 862b61697a984e089623807d56a09ae3; no voters:
I20260810 18:58:08.472034 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:58:08.472290 415 leader_election.cc:288] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:58:08.472357 423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:08.472589 423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [term 1 LEADER]: Becoming Leader. State: Replica: 862b61697a984e089623807d56a09ae3, State: Running, Role: LEADER
I20260810 18:58:08.472939 423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b61697a984e089623807d56a09ae3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34085 } }
I20260810 18:58:08.473562 415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:08.474179 423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 862b61697a984e089623807d56a09ae3. Latest consensus state: current_term: 1 leader_uuid: "862b61697a984e089623807d56a09ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b61697a984e089623807d56a09ae3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34085 } } }
I20260810 18:58:08.474324 423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:08.474704 424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "862b61697a984e089623807d56a09ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "862b61697a984e089623807d56a09ae3" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 34085 } } }
I20260810 18:58:08.474843 424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:08.474824 439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:08.478273 439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:08.481172 418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:08.476139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54935 (local address 127.30.153.190:34085)
0810 18:58:08.476262 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:08.476267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:08.476565 (+ 298us) server_negotiation.cc:407] Connection header received
0810 18:58:08.476755 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:08.476758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:08.476803 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:08.476853 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:08.477231 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:08.478101 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:08.479292 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:08.479650 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:08.479780 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:08.480676 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:08.480681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:08.480684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:08.480699 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:08.480731 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:08.480748 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:08.480842 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:08.480948 (+ 106us) server_negotiation.cc:299] Negotiation successful
0810 18:58:08.481009 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260810 18:58:08.484035 439 catalog_manager.cc:1347] Generated new cluster ID: 6aeeb87d4b3b4733a3128a2371d4465f
I20260810 18:58:08.484148 439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:08.525509 439 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:08.526846 439 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:08.539922 439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 862b61697a984e089623807d56a09ae3: Generated new TSK 0
I20260810 18:58:08.540627 439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:58:08.552893 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 311
2026-08-10T18:58:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1686 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:58:08 dist-test-slave-68xk krb5kdc[462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:58:08 dist-test-slave-68xk krb5kdc[462](info): set up 2 sockets
Aug 10 18:58:08 dist-test-slave-68xk krb5kdc[462](info): commencing operation
krb5kdc: starting...
W20260810 18:58:09.801520 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:58:09 dist-test-slave-68xk krb5kdc[462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:58:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:58:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:58:09.937914 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:32805
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:32805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:10.052325 486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:10.052634 486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:10.052757 486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:10.056816 486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:10.056921 486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:10.056962 486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:10.057018 486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:10.057058 486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:10.062230 486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:32805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:10.063323 486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:10.064375 486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:10.073001 500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:10.072989 503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:10.073024 495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:10.073272 486 server_base.cc:1034] running on GCE node
I20260810 18:58:10.073799 486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:10.074994 486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:10.076184 486 hybrid_clock.cc:648] HybridClock initialized: now 1786388290076158 us; error 39 us; skew 500 ppm
Aug 10 18:58:10 dist-test-slave-68xk krb5kdc[462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388290, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:10.079046 486 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:58:10.079922 486 webserver.cc:460] Webserver started at http://127.30.153.190:41367/ using document root <none> and password file <none>
I20260810 18:58:10.080432 486 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:10.080492 486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:10.080653 486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:10.082751 486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "b792f32f178f414e829bef5c576a0d0b"
format_stamp: "Formatted at 2026-08-10 18:58:10 on dist-test-slave-68xk"
server_key: "a33efd6b0bc90348c7d3bb7a1b7fc769"
server_key_iv: "acf280e555ea0456d4e0677299915b64"
server_key_version: "encryptionkey@0"
I20260810 18:58:10.083293 486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "b792f32f178f414e829bef5c576a0d0b"
format_stamp: "Formatted at 2026-08-10 18:58:10 on dist-test-slave-68xk"
server_key: "a33efd6b0bc90348c7d3bb7a1b7fc769"
server_key_iv: "acf280e555ea0456d4e0677299915b64"
server_key_version: "encryptionkey@0"
I20260810 18:58:10.087009 486 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 18:58:10.089375 513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:10.090718 486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:58:10.090871 486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "b792f32f178f414e829bef5c576a0d0b"
format_stamp: "Formatted at 2026-08-10 18:58:10 on dist-test-slave-68xk"
server_key: "a33efd6b0bc90348c7d3bb7a1b7fc769"
server_key_iv: "acf280e555ea0456d4e0677299915b64"
server_key_version: "encryptionkey@0"
I20260810 18:58:10.090975 486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:10.095643 486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:10.099545 486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:10.099772 486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:10.107311 486 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:32805
I20260810 18:58:10.107343 601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:32805 every 8 connection(s)
I20260810 18:58:10.108335 486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:10.111308 603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:10.115267 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 486
I20260810 18:58:10.115417 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:10.115727 31334 external_mini_cluster.cc:1428] Setting key 8914d74121e32962edf991503155ed43
I20260810 18:58:10.117443 603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b: Bootstrap starting.
I20260810 18:58:10.120406 603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:10.121333 603 log.cc:824] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:10.121560 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:10.117093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54845 (local address 127.30.153.190:32805)
0810 18:58:10.117544 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:10.117565 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:10.117609 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:58:10.118484 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:10.118506 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:10.118917 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:10.119294 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:10.120131 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:10.120136 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:10.120151 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:10.120163 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:10.120232 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:10.120264 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:10.120388 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:10.120477 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 18:58:10.120644 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":139,"threads_started":1}
I20260810 18:58:10.124066 603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b: No bootstrap required, opened a new log
I20260810 18:58:10.124605 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:58:10.128654 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:10.126804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43707 (local address 127.30.153.190:32805)
0810 18:58:10.126940 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:10.126944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:10.127166 (+ 222us) server_negotiation.cc:407] Connection header received
0810 18:58:10.127346 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:10.127348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:10.127378 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:10.127445 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:10.128162 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:10.128165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:10.128167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 18:58:10.128177 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:10.128201 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 18:58:10.128211 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:10.128286 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:10.128395 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 18:58:10.128429 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 18:58:10.128489 603 raft_consensus.cc:348] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792f32f178f414e829bef5c576a0d0b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 32805 } }
I20260810 18:58:10.128796 603 raft_consensus.cc:374] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 18:58:10.128851 603 raft_consensus.cc:729] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b792f32f178f414e829bef5c576a0d0b, State: Initialized, Role: FOLLOWER
I20260810 18:58:10.129307 603 consensus_queue.cc:260] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792f32f178f414e829bef5c576a0d0b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 32805 } }
I20260810 18:58:10.129472 603 raft_consensus.cc:388] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 18:58:10.129547 603 raft_consensus.cc:482] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 18:58:10.129666 603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:10.130739 603 raft_consensus.cc:504] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792f32f178f414e829bef5c576a0d0b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 32805 } }
I20260810 18:58:10.131091 603 leader_election.cc:302] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b792f32f178f414e829bef5c576a0d0b; no voters:
I20260810 18:58:10.131415 603 leader_election.cc:288] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 18:58:10.131526 623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:10.131772 623 raft_consensus.cc:686] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [term 1 LEADER]: Becoming Leader. State: Replica: b792f32f178f414e829bef5c576a0d0b, State: Running, Role: LEADER
I20260810 18:58:10.132099 623 consensus_queue.cc:237] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792f32f178f414e829bef5c576a0d0b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 32805 } }
I20260810 18:58:10.132676 603 sys_catalog.cc:564] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:10.134220 625 sys_catalog.cc:455] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b792f32f178f414e829bef5c576a0d0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792f32f178f414e829bef5c576a0d0b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 32805 } } }
I20260810 18:58:10.134366 625 sys_catalog.cc:458] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:10.134652 627 sys_catalog.cc:455] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b792f32f178f414e829bef5c576a0d0b. Latest consensus state: current_term: 1 leader_uuid: "b792f32f178f414e829bef5c576a0d0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b792f32f178f414e829bef5c576a0d0b" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 32805 } } }
I20260810 18:58:10.134749 627 sys_catalog.cc:458] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b [sys.catalog]: This master's current role is: LEADER
W20260810 18:58:10.137421 646 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:10.137517 646 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:10.137584 644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:10.138301 644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:10.143687 644 catalog_manager.cc:1347] Generated new cluster ID: 653c8591c9174bde93f117463841808a
I20260810 18:58:10.143751 644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:10.157477 644 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:10.157838 644 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:10.164932 644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b792f32f178f414e829bef5c576a0d0b: Generated new TSK 0
I20260810 18:58:10.165800 644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 18:58:10.171633 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 486
2026-08-10T18:58:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1614 ms)
[----------] 18 tests from AuthTokenIssuingTest (30094 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:58:10 dist-test-slave-68xk krb5kdc[656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:58:10 dist-test-slave-68xk krb5kdc[656](info): set up 2 sockets
Aug 10 18:58:10 dist-test-slave-68xk krb5kdc[656](info): commencing operation
krb5kdc: starting...
W20260810 18:58:11.485739 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:58:11 dist-test-slave-68xk krb5kdc[656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:58:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:58:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:58:11.622777 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:35507
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:35507,127.30.153.189:44899,127.30.153.188:33963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:11.735786 682 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:58:11.736119 682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:11.736184 682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:11.736265 682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:11.740259 682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:11.740345 682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:11.740384 682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:11.740422 682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:11.740456 682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:11.745654 682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:35507,127.30.153.189:44899,127.30.153.188:33963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:11.746809 682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:11.747881 682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:11.757169 695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:11.757169 693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:11.757229 698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:11.757318 682 server_base.cc:1034] running on GCE node
I20260810 18:58:11.758067 682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:11.759209 682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:11.760411 682 hybrid_clock.cc:648] HybridClock initialized: now 1786388291760395 us; error 28 us; skew 500 ppm
Aug 10 18:58:11 dist-test-slave-68xk krb5kdc[656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388291, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:11.763360 682 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:58:11.764407 682 webserver.cc:460] Webserver started at http://127.30.153.190:39265/ using document root <none> and password file <none>
I20260810 18:58:11.765076 682 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:11.765182 682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:11.765448 682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:11.767817 682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "e9812640601042e9974eff22d1646dfe"
format_stamp: "Formatted at 2026-08-10 18:58:11 on dist-test-slave-68xk"
server_key: "7f85b58d577a92b1a97a760767648512"
server_key_iv: "5aab68078efac5836e757b34024dcb7f"
server_key_version: "encryptionkey@0"
I20260810 18:58:11.768507 682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "e9812640601042e9974eff22d1646dfe"
format_stamp: "Formatted at 2026-08-10 18:58:11 on dist-test-slave-68xk"
server_key: "7f85b58d577a92b1a97a760767648512"
server_key_iv: "5aab68078efac5836e757b34024dcb7f"
server_key_version: "encryptionkey@0"
I20260810 18:58:11.772652 682 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:58:11.775346 712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:11.776569 682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:58:11.776722 682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "e9812640601042e9974eff22d1646dfe"
format_stamp: "Formatted at 2026-08-10 18:58:11 on dist-test-slave-68xk"
server_key: "7f85b58d577a92b1a97a760767648512"
server_key_iv: "5aab68078efac5836e757b34024dcb7f"
server_key_version: "encryptionkey@0"
I20260810 18:58:11.776836 682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:11.815181 682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:11.819481 682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:11.819705 682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:11.827327 682 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:35507
I20260810 18:58:11.827637 792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:35507 every 8 connection(s)
I20260810 18:58:11.828346 682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:11.830652 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 682
I20260810 18:58:11.830799 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:11.831104 31334 external_mini_cluster.cc:1428] Setting key 55af9fa77d50b89b83505c2d4d4eaf38
I20260810 18:58:11.831573 793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:11.835530 793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.153.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.189@KRBTEST.COM" created.
Aug 10 18:58:11 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388291, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:58:11.852811 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:11.837959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:35507 (local address 127.0.0.1:46680)
0810 18:58:11.838501 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:11.838550 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:11.839426 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:11.840284 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:11.840288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:11.840609 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:11.840899 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:11.840913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:11.842108 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:11.842111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:11.843315 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:11.843326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:11.844252 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:11.844255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:11.844348 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:11.845045 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:11.845076 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:11.847157 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:11.849826 (+ 2669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:11.849834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:11.849842 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:11.850069 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:11.850511 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:11.850513 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:11.850515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:11.850626 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:11.851470 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:11.851475 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:11.851603 (+ 128us) client_negotiation.cc:769] Sending connection context
0810 18:58:11.851735 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 18:58:11.852005 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":150,"threads_started":1}
I20260810 18:58:11.852814 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:11.837959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46680 (local address 127.30.153.190:35507)
0810 18:58:11.838551 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:11.838570 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:11.838682 (+ 112us) server_negotiation.cc:407] Connection header received
0810 18:58:11.839764 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:11.839773 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:11.840031 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:11.840218 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:11.841048 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:11.841928 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:11.843445 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:11.844079 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:11.844293 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:11.847341 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:11.847347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:11.847358 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:11.847382 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:11.849692 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:11.850210 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:11.850219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:11.850222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:11.850378 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:11.850743 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:11.850749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:11.850752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:11.851346 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:11.851504 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:11.851885 (+ 381us) server_negotiation.cc:299] Negotiation successful
0810 18:58:11.852104 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":460,"spinlock_wait_cycles":12544,"thread_start_us":147,"threads_started":1}
I20260810 18:58:11.855473 793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:11.856463 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:11.856015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38612)
0810 18:58:11.856211 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:11.856371 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260810 18:58:11.856801 717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
I20260810 18:58:11.857910 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:11.857676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38626)
0810 18:58:11.857807 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:11.857856 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 18:58:11.858142 793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:44899: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab.
Entry for principal kudu/127.30.153.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab.
I20260810 18:58:11.862883 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.189:44899
--webserver_interface=127.30.153.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab
--principal=kudu/127.30.153.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:35507,127.30.153.189:44899,127.30.153.188:33963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:58:11.907477 793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } attempt: 1
I20260810 18:58:11.908492 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:11.908160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38642)
0810 18:58:11.908345 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:11.908409 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 18:58:11.909233 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:11.909012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38656)
0810 18:58:11.909122 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:11.909177 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:58:11.909358 793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:44899: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
I20260810 18:58:11.977573 793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } attempt: 2
I20260810 18:58:11.978585 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:11.978193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38662)
0810 18:58:11.978379 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:11.978505 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 18:58:11.979403 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:11.979125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38670)
0810 18:58:11.979239 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:11.979325 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 18:58:11.979538 793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:44899: Network error: Client connection negotiation failed: client connection to 127.30.153.189:44899: connect: Connection refused (error 111)
W20260810 18:58:11.980489 800 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:58:11.980827 800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:11.980877 800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:11.980948 800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:11.985025 800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:11.985096 800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:11.985116 800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:11.985169 800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:11.985203 800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:11.990592 800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:35507,127.30.153.189:44899,127.30.153.188:33963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab
--principal=kudu/127.30.153.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.189:44899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.153.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:11.991626 800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:11.992674 800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:12.001477 806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:12.001462 805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:12.001482 808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:12.001823 800 server_base.cc:1034] running on GCE node
I20260810 18:58:12.002573 800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:12.003721 800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:12.004931 800 hybrid_clock.cc:648] HybridClock initialized: now 1786388292004916 us; error 30 us; skew 500 ppm
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:12.007951 800 init.cc:377] Logged in from keytab as kudu/127.30.153.189@KRBTEST.COM (short username kudu)
I20260810 18:58:12.008865 800 webserver.cc:460] Webserver started at http://127.30.153.189:37219/ using document root <none> and password file <none>
I20260810 18:58:12.009451 800 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:12.009549 800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:12.009801 800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:12.011955 800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/data/instance:
uuid: "bf53bfc315694551b6725c68326c4d95"
format_stamp: "Formatted at 2026-08-10 18:58:12 on dist-test-slave-68xk"
server_key: "abec87fe268346b5f10b445b8ef6aabd"
server_key_iv: "45aa61b0d803aba2dde07f24c42a4a1a"
server_key_version: "encryptionkey@0"
I20260810 18:58:12.012532 800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/wal/instance:
uuid: "bf53bfc315694551b6725c68326c4d95"
format_stamp: "Formatted at 2026-08-10 18:58:12 on dist-test-slave-68xk"
server_key: "abec87fe268346b5f10b445b8ef6aabd"
server_key_iv: "45aa61b0d803aba2dde07f24c42a4a1a"
server_key_version: "encryptionkey@0"
I20260810 18:58:12.016434 800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 18:58:12.018875 818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:12.020169 800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:58:12.020330 800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/wal
uuid: "bf53bfc315694551b6725c68326c4d95"
format_stamp: "Formatted at 2026-08-10 18:58:12 on dist-test-slave-68xk"
server_key: "abec87fe268346b5f10b445b8ef6aabd"
server_key_iv: "45aa61b0d803aba2dde07f24c42a4a1a"
server_key_version: "encryptionkey@0"
I20260810 18:58:12.020448 800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:12.036890 800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:12.040123 800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:12.040386 800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:12.047853 800 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.189:44899
I20260810 18:58:12.048177 908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.189:44899 every 8 connection(s)
I20260810 18:58:12.048866 800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
I20260810 18:58:12.050773 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 800
I20260810 18:58:12.050943 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-1/wal/instance
I20260810 18:58:12.051283 31334 external_mini_cluster.cc:1428] Setting key 81c6add40ca96c9fdb216e71a4dc8097
I20260810 18:58:12.052393 910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:12.056833 910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.153.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.188@KRBTEST.COM" created.
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:58:12.070748 793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } attempt: 3
I20260810 18:58:12.073514 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.058902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.30.153.190:35507)
0810 18:58:12.059062 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.059066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.060121 (+ 1055us) server_negotiation.cc:407] Connection header received
0810 18:58:12.061193 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.061197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.061301 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.061403 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.062785 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.063525 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.064813 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.065236 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.065365 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.068253 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.068260 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.068266 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.068316 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.070996 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.071660 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.071665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.071668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.071763 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.072242 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.072246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.072247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.072542 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.072660 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.073205 (+ 545us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.073340 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 18:58:12.074601 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.059392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:35507 (local address 127.0.0.1:46694)
0810 18:58:12.059956 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.060001 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:12.060954 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:12.061620 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:12.061627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:12.061975 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:12.062589 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.062637 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.063674 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.063678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.064662 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.064672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.065426 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.065429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.065579 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.066250 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:12.066266 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:12.068052 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:12.071134 (+ 3082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.071143 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.071162 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.071537 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.071911 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.071914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.071916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.072120 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.072684 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:12.072690 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:12.072918 (+ 228us) client_negotiation.cc:769] Sending connection context
0810 18:58:12.073122 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 18:58:12.073440 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":176,"threads_started":1}
I20260810 18:58:12.076202 910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388291, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab.
Entry for principal kudu/127.30.153.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab.
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
I20260810 18:58:12.082139 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.188:33963
--webserver_interface=127.30.153.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab
--principal=kudu/127.30.153.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:38887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:35507,127.30.153.189:44899,127.30.153.188:33963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:58:12.083328 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.071314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38676)
0810 18:58:12.071765 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.071783 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:12.071858 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:12.072557 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:12.072562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:12.072794 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:12.072921 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.072928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.074276 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.074280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.075025 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.075035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.075856 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.075859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.075956 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.076406 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:12.076418 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:12.078022 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:12.080404 (+ 2382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.080408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.080410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.080691 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.081837 (+ 1146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.081843 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.081848 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.081961 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.082649 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:12.082655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:12.082789 (+ 134us) client_negotiation.cc:769] Sending connection context
0810 18:58:12.082885 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 18:58:12.083060 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339}
I20260810 18:58:12.084143 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.071615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38676 (local address 127.30.153.189:44899)
0810 18:58:12.072070 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.072079 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.072100 (+ 21us) server_negotiation.cc:407] Connection header received
0810 18:58:12.072134 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.072138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.072384 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.072533 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.073054 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.074161 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.075179 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.075727 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.075834 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.078144 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.078153 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.078163 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.078183 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.080300 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.081567 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.081573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.081575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.081668 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.082091 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.082094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.082096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.082515 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.083567 (+ 1052us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.083695 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.083877 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":140,"threads_started":1}
I20260810 18:58:12.085945 793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:12.086812 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.086537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47832)
0810 18:58:12.086669 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.086730 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 18:58:12.087566 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.076753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38678)
0810 18:58:12.076847 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.076863 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:12.076945 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:12.077435 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:12.077437 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:12.077637 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:12.077781 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.077787 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.078787 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.078793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.079655 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.079669 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.080302 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.080309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.080426 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.080898 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:12.080909 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:12.082547 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:12.085656 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.085661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.085666 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.085935 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.086305 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.086310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.086312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.086404 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.086897 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:12.086902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:12.087022 (+ 120us) client_negotiation.cc:769] Sending connection context
0810 18:58:12.087146 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 18:58:12.087311 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260810 18:58:12.087630 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.076810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.30.153.189:44899)
0810 18:58:12.077188 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.077198 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.077212 (+ 14us) server_negotiation.cc:407] Connection header received
0810 18:58:12.077245 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.077248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.077301 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.077409 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.077900 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.078661 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.079793 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.080156 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.080279 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.082736 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.082739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.082741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.082762 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.085511 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.086072 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.086075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.086077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.086194 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.086550 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.086553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.086555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.086774 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.086890 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.087214 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.087396 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":307,"thread_start_us":107,"threads_started":1}
I20260810 18:58:12.087679 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.087362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47844)
0810 18:58:12.087554 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.087607 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260810 18:58:12.087849 793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:33963: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
I20260810 18:58:12.088678 910 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:12.089465 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.089144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47856)
0810 18:58:12.089283 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.089382 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 18:58:12.089715 822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
I20260810 18:58:12.090679 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.090359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47868)
0810 18:58:12.090526 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.090593 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260810 18:58:12.090870 910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:33963: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
I20260810 18:58:12.119040 793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } attempt: 1
I20260810 18:58:12.119942 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.119623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47874)
0810 18:58:12.119794 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.119858 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 18:58:12.120730 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.120491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47876)
0810 18:58:12.120603 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.120676 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:58:12.120873 793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:33963: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
I20260810 18:58:12.140149 910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } attempt: 1
I20260810 18:58:12.141055 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.140751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47886)
0810 18:58:12.140913 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.140976 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 18:58:12.141889 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.141632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47900)
0810 18:58:12.141767 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.141831 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260810 18:58:12.142066 910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:33963: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
I20260810 18:58:12.196089 793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } attempt: 2
I20260810 18:58:12.196954 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.196642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47902)
0810 18:58:12.196796 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.196871 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:58:12.197731 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.197510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47908)
0810 18:58:12.197625 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.197677 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:58:12.197896 793 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:33963: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
W20260810 18:58:12.201301 919 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:58:12.201591 919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:12.201638 919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:12.201709 919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:12.205863 919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:12.205917 919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:12.205937 919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:12.205983 919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:12.206022 919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:12.210239 910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } attempt: 2
I20260810 18:58:12.211097 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.210771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47924)
0810 18:58:12.210944 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.211011 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 18:58:12.211658 919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:38887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:35507,127.30.153.189:44899,127.30.153.188:33963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab
--principal=kudu/127.30.153.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.188:33963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.153.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:12.212028 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.211737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47938)
0810 18:58:12.211902 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.211958 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260810 18:58:12.212241 910 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:33963: Network error: Client connection negotiation failed: client connection to 127.30.153.188:33963: connect: Connection refused (error 111)
I20260810 18:58:12.212865 919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:12.213899 919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:12.222326 932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:12.222329 928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:12.222357 930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:12.222762 919 server_base.cc:1034] running on GCE node
I20260810 18:58:12.223335 919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:12.224491 919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:12.225674 919 hybrid_clock.cc:648] HybridClock initialized: now 1786388292225651 us; error 46 us; skew 500 ppm
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:12.228521 919 init.cc:377] Logged in from keytab as kudu/127.30.153.188@KRBTEST.COM (short username kudu)
I20260810 18:58:12.229476 919 webserver.cc:460] Webserver started at http://127.30.153.188:33657/ using document root <none> and password file <none>
I20260810 18:58:12.230051 919 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:12.230115 919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:12.230392 919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:12.232498 919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/data/instance:
uuid: "6a6453531bce4feabad062bcfccaf0c6"
format_stamp: "Formatted at 2026-08-10 18:58:12 on dist-test-slave-68xk"
server_key: "6978edf4368edbce032a863d955a80ad"
server_key_iv: "1a1eb69da2e12c4bef9eb4ba671b4add"
server_key_version: "encryptionkey@0"
I20260810 18:58:12.233088 919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/wal/instance:
uuid: "6a6453531bce4feabad062bcfccaf0c6"
format_stamp: "Formatted at 2026-08-10 18:58:12 on dist-test-slave-68xk"
server_key: "6978edf4368edbce032a863d955a80ad"
server_key_iv: "1a1eb69da2e12c4bef9eb4ba671b4add"
server_key_version: "encryptionkey@0"
I20260810 18:58:12.236901 919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260810 18:58:12.239414 944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:12.240599 919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:58:12.240748 919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/wal
uuid: "6a6453531bce4feabad062bcfccaf0c6"
format_stamp: "Formatted at 2026-08-10 18:58:12 on dist-test-slave-68xk"
server_key: "6978edf4368edbce032a863d955a80ad"
server_key_iv: "1a1eb69da2e12c4bef9eb4ba671b4add"
server_key_version: "encryptionkey@0"
I20260810 18:58:12.240860 919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:12.256804 919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:12.259649 919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:12.259853 919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:12.267019 919 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.188:33963
I20260810 18:58:12.267321 1019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.188:33963 every 8 connection(s)
I20260810 18:58:12.268006 919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
I20260810 18:58:12.270813 1021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:12.271883 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 919
I20260810 18:58:12.272022 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786387990686100-31334-0/minicluster-data/master-2/wal/instance
I20260810 18:58:12.272357 31334 external_mini_cluster.cc:1428] Setting key 4352c7de1ca4f1e42900ac17bf70aa87
I20260810 18:58:12.275310 1021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:58:12.286804 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.277019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.30.153.190:35507)
0810 18:58:12.277170 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.277177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.277488 (+ 311us) server_negotiation.cc:407] Connection header received
0810 18:58:12.277655 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.277658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.277714 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.277802 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.278458 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.278948 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.279849 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.280151 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.280272 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.282899 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.282905 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.282907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.282940 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.284464 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.284982 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.284986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.284988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.285098 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.285487 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.285490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.285492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.285752 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.285855 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.286270 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.286417 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 18:58:12.287468 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.273765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47954 (local address 127.30.153.188:33963)
0810 18:58:12.274206 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.274221 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.274256 (+ 35us) server_negotiation.cc:407] Connection header received
0810 18:58:12.274956 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.274973 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.275297 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.275626 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.276490 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.277900 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.278861 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.279409 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.279591 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.282330 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.282339 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.282354 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.282376 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.284416 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.284992 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.284998 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.284999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.285139 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.285487 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.285490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.285492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.285973 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.286133 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.286434 (+ 301us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.286636 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":294,"spinlock_wait_cycles":2304,"thread_start_us":140,"threads_started":1}
I20260810 18:58:12.287467 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.276955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:35507 (local address 127.0.0.1:46706)
0810 18:58:12.277359 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.277398 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:12.277545 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:12.277853 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:12.277859 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:12.278192 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:12.278311 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.278316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.279071 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.279077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.279724 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.279733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.280329 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.280334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.280443 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.280998 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:12.281014 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:12.282715 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:12.284571 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.284581 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.284591 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.284882 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.285196 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.285198 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.285199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.285385 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.285859 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:12.285864 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:12.285995 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 18:58:12.286191 (+ 196us) client_negotiation.cc:240] Negotiation successful
0810 18:58:12.286568 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":1,"spinlock_wait_cycles":1408,"thread_start_us":159,"threads_started":1}
I20260810 18:58:12.288681 1021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:58:12.297070 793 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } attempt: 3
I20260810 18:58:12.299611 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.289187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:44899 (local address 127.0.0.1:38690)
0810 18:58:12.289671 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.289684 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:12.289789 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:12.290096 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:12.290098 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:12.290334 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:12.290508 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.290516 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.291511 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.291515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.292142 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.292151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.292787 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.292790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.292887 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.293529 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:12.293557 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:12.295138 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:12.297680 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.297685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.297687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.297968 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.298372 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.298376 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.298381 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.298558 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.299069 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:12.299073 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:12.299169 (+ 96us) client_negotiation.cc:769] Sending connection context
0810 18:58:12.299278 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 18:58:12.299412 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260810 18:58:12.300005 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.289322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.30.153.189:44899)
0810 18:58:12.289450 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.289454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.289762 (+ 308us) server_negotiation.cc:407] Connection header received
0810 18:58:12.289908 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.289911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.289984 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.290064 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.290644 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.291401 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.292266 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.292666 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.292776 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.295320 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.295324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.295326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.295352 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.297528 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.298119 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.298125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.298138 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.298251 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.298683 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.298687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.298689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.298894 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.299022 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.299395 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.299548 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 18:58:12.300606 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.289942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.30.153.190:35507)
0810 18:58:12.290049 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.290054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.290139 (+ 85us) server_negotiation.cc:407] Connection header received
0810 18:58:12.290264 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.290270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.290319 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.290406 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.291167 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.291761 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.292700 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.293090 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.293219 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.296032 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.296036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.296039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.296067 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.297940 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.298577 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.298583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.298588 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.298682 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.299045 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.299048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.299049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.299233 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.299514 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.300279 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.300419 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:58:12.300761 1021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388291, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
I20260810 18:58:12.303419 910 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } attempt: 3
I20260810 18:58:12.308681 796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.297653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47968)
0810 18:58:12.298037 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.298058 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:12.298143 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:12.298530 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:12.298535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:12.298736 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:12.298853 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.298862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.299922 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.299927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.300762 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.300775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.301442 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.301446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.301576 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.302188 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:12.302206 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:12.304449 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:12.306779 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.306782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.306785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.307074 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.307411 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.307414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.307416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.307526 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.307979 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:12.307983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:12.308121 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 18:58:12.308272 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 18:58:12.308452 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260810 18:58:12.309281 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.297864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47968 (local address 127.30.153.188:33963)
0810 18:58:12.298046 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.298053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.298224 (+ 171us) server_negotiation.cc:407] Connection header received
0810 18:58:12.298255 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.298257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.298292 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.298477 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.299002 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.299818 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.300913 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.301286 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.302547 (+ 1261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.304593 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.304599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.304602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.304632 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.306649 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.307209 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.307213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.307215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.307302 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.307637 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.307641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.307642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.307878 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.307982 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.308943 (+ 961us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.309091 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":67}
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
Aug 10 18:58:12 dist-test-slave-68xk krb5kdc[656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
I20260810 18:58:12.313632 793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe: Bootstrap starting.
I20260810 18:58:12.315738 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.304003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47988)
0810 18:58:12.304205 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.304223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:12.304309 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:12.304655 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:12.304660 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:12.304876 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:12.305002 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.305010 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.306489 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.306494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.307411 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.307422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.308182 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.308187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.308283 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.308722 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:12.308762 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:12.310866 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:12.313668 (+ 2802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.313674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.313676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.313964 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.314344 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.314348 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.314350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.314524 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.315055 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:12.315062 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:12.315197 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 18:58:12.315310 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 18:58:12.315502 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 18:58:12.315827 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.301274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:33963 (local address 127.0.0.1:47978)
0810 18:58:12.304237 (+ 2963us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:12.304253 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:12.304345 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:12.304718 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:12.304721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:12.304921 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:12.305001 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.305007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.306142 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.306145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.307007 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:12.307018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.307799 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.307805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:12.309472 (+ 1667us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.309982 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:12.309998 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:12.311562 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:12.314167 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.314174 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.314179 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.314484 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.314788 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:12.314791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:12.314793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:12.314877 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:12.315366 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:12.315369 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:12.315452 (+ 83us) client_negotiation.cc:769] Sending connection context
0810 18:58:12.315542 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 18:58:12.315670 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2898,"mutex_wait_us":1539}
I20260810 18:58:12.316382 1027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.302661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47978 (local address 127.30.153.188:33963)
0810 18:58:12.303836 (+ 1175us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.303844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.304429 (+ 585us) server_negotiation.cc:407] Connection header received
0810 18:58:12.304460 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.304463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.304510 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.304599 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.305157 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.306009 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.307156 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.307595 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.307725 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.311705 (+ 3980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.311709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.311712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.311744 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.313999 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.314598 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.314601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.314602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.314687 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.314971 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.314974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.314976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.315227 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.315329 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.316074 (+ 745us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.316205 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":255,"server-negotiator.queue_time_us":1108,"thread_start_us":1423,"threads_started":1}
I20260810 18:58:12.316593 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.304034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47988 (local address 127.30.153.188:33963)
0810 18:58:12.304359 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.304371 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.304386 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:58:12.304423 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.304426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.304472 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.305167 (+ 695us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.305316 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.306298 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.307576 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.308064 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.308165 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.311068 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.311071 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.311073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.311097 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.313509 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.314139 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.314142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.314144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.314233 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.314649 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.314652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.314655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.314897 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.316090 (+ 1193us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.316174 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.316303 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":253,"thread_start_us":135,"threads_started":1}
I20260810 18:58:12.317132 793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:12.319561 793 log.cc:824] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:12.319640 910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95: Bootstrap starting.
I20260810 18:58:12.319640 1021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6: Bootstrap starting.
I20260810 18:58:12.320155 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.301879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.30.153.189:44899)
0810 18:58:12.301991 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.301995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.302081 (+ 86us) server_negotiation.cc:407] Connection header received
0810 18:58:12.302280 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.302283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.302332 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.302391 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.304613 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.305449 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.306705 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.307073 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.314569 (+ 7496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.314691 (+ 122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.314697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.314704 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.314725 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.317500 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.318000 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.318004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.318006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.318099 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.318446 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.318454 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.318467 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.318719 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.319623 (+ 904us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.319845 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.319980 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:58:12.322693 910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:12.322692 1021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:12.322968 793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe: No bootstrap required, opened a new log
I20260810 18:58:12.323704 910 log.cc:824] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:12.323702 1021 log.cc:824] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:12.324096 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:58:12.325785 759 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e9812640601042e9974eff22d1646dfe"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46714
I20260810 18:58:12.327498 793 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } }
I20260810 18:58:12.327731 793 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9812640601042e9974eff22d1646dfe, State: Initialized, Role: FOLLOWER
I20260810 18:58:12.328034 793 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } }
I20260810 18:58:12.328375 910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95: No bootstrap required, opened a new log
I20260810 18:58:12.328737 1035 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.328965 1035 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:12.329589 793 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:12.329646 759 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 18:58:12.329777 759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:12.331143 759 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } }
I20260810 18:58:12.329551 1021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6: No bootstrap required, opened a new log
I20260810 18:58:12.333278 910 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } }
I20260810 18:58:12.333534 910 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf53bfc315694551b6725c68326c4d95, State: Initialized, Role: FOLLOWER
I20260810 18:58:12.333864 910 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } }
I20260810 18:58:12.332104 759 leader_election.cc:288] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [CANDIDATE]: Term 1 election: Requested vote from peers bf53bfc315694551b6725c68326c4d95 (127.30.153.189:44899), 6a6453531bce4feabad062bcfccaf0c6 (127.30.153.188:33963)
I20260810 18:58:12.335306 910 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:12.335490 863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9812640601042e9974eff22d1646dfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bf53bfc315694551b6725c68326c4d95"
I20260810 18:58:12.335721 863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:12.336998 1021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } }
I20260810 18:58:12.337299 1021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a6453531bce4feabad062bcfccaf0c6, State: Initialized, Role: FOLLOWER
I20260810 18:58:12.337617 1021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } }
I20260810 18:58:12.337836 863 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9812640601042e9974eff22d1646dfe in term 1.
I20260810 18:58:12.338447 717 leader_election.cc:302] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf53bfc315694551b6725c68326c4d95, e9812640601042e9974eff22d1646dfe; no voters:
I20260810 18:58:12.338709 1035 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:12.338922 1035 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [term 1 LEADER]: Becoming Leader. State: Replica: e9812640601042e9974eff22d1646dfe, State: Running, Role: LEADER
I20260810 18:58:12.338999 1021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:12.339226 1035 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } }
I20260810 18:58:12.339412 1086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.339533 1086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:12.341051 1035 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9812640601042e9974eff22d1646dfe. Latest consensus state: current_term: 1 leader_uuid: "e9812640601042e9974eff22d1646dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.341171 1035 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: This master's current role is: LEADER
W20260810 18:58:12.341768 1058 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:12.341866 1058 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:12.341951 1107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:12.338739 682 master_runner.cc:186] Error getting master registration for 127.30.153.189:44899: OK, instance_id { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" instance_seqno: 1786388292045670 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:58:12.342145 682 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:58:12.343001 1107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:12.342257 982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9812640601042e9974eff22d1646dfe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6a6453531bce4feabad062bcfccaf0c6"
I20260810 18:58:12.343420 982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:12.344874 982 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9812640601042e9974eff22d1646dfe in term 1.
W20260810 18:58:12.346649 1105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:12.346742 1105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:12.348364 982 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [term 1 FOLLOWER]: Refusing update from remote peer e9812640601042e9974eff22d1646dfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:58:12.349046 1035 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:58:12.350565 863 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [term 1 FOLLOWER]: Refusing update from remote peer e9812640601042e9974eff22d1646dfe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:58:12.351166 1090 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:58:12.353318 1086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9812640601042e9974eff22d1646dfe. Latest consensus state: current_term: 1 leader_uuid: "e9812640601042e9974eff22d1646dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.353477 1086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 18:58:12.358218 1087 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:12.358311 1087 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:12.363023 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.349453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47996 (local address 127.30.153.188:33963)
0810 18:58:12.349576 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.349583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.352984 (+ 3401us) server_negotiation.cc:407] Connection header received
0810 18:58:12.353118 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.353122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.353182 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.353282 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.354498 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.355317 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.356576 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.357045 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.357143 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.359238 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.359243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.359246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.359272 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.361061 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.361601 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.361605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.361607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.361697 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.362013 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.362017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.362019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.362242 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.362358 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.362752 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.362874 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 18:58:12.367401 1035 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9812640601042e9974eff22d1646dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.367548 1035 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:12.367662 1035 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9812640601042e9974eff22d1646dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.367749 1035 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:12.368896 1086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9812640601042e9974eff22d1646dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.369019 1086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:12.369024 1107 catalog_manager.cc:1347] Generated new cluster ID: 8b109e83ce734a5d9bbc61c2565a5910
I20260810 18:58:12.369082 1107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:12.370716 1112 mvcc.cc:205] Tried to move back new op lower bound from 7317046445452541952 to 7317046445424861184. Current Snapshot: MvccSnapshot[applied={T|T < 7317046445452541952 or (T in {7317046445452541952})}]
I20260810 18:58:12.371075 1117 mvcc.cc:205] Tried to move back new op lower bound from 7317046445452541952 to 7317046445424861184. Current Snapshot: MvccSnapshot[applied={T|T < 7317046445452541952}]
I20260810 18:58:12.371732 1132 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9812640601042e9974eff22d1646dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.371866 1132 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:12.371949 1132 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9812640601042e9974eff22d1646dfe. Latest consensus state: current_term: 1 leader_uuid: "e9812640601042e9974eff22d1646dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.372009 1132 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:12.372067 1132 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9812640601042e9974eff22d1646dfe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9812640601042e9974eff22d1646dfe" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 35507 } } peers { permanent_uuid: "bf53bfc315694551b6725c68326c4d95" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 44899 } } peers { permanent_uuid: "6a6453531bce4feabad062bcfccaf0c6" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 33963 } } }
I20260810 18:58:12.372121 1132 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:12.372256 1120 mvcc.cc:205] Tried to move back new op lower bound from 7317046445452541952 to 7317046445424861184. Current Snapshot: MvccSnapshot[applied={T|T < 7317046445452541952 or (T in {7317046445452541952})}]
I20260810 18:58:12.376633 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.345597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.30.153.190:35507)
0810 18:58:12.345918 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.345923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.352760 (+ 6837us) server_negotiation.cc:407] Connection header received
0810 18:58:12.353247 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.353250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.353311 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.353364 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.354745 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.355544 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.359789 (+ 4245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.360120 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.360201 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.364669 (+ 4468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.364673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.364676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.364707 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.366583 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.374877 (+ 8294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.374881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.374883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.374992 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.375344 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.375348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.375350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.375573 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.375690 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.376352 (+ 662us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.376477 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260810 18:58:12.379652 918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:12.350242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.30.153.189:44899)
0810 18:58:12.350361 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.350366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.354057 (+ 3691us) server_negotiation.cc:407] Connection header received
0810 18:58:12.356847 (+ 2790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.356850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.356900 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.356950 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.357912 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.358663 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.361939 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:12.362295 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:12.362383 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:12.375775 (+ 13392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:12.375779 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:12.375782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:12.375814 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:12.377664 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.378210 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.378214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.378216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.378311 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:12.378676 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:12.378680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:12.378681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:12.378898 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:12.378997 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:12.379371 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 18:58:12.379496 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 18:58:12.410828 1107 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:12.412199 1107 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:12.430292 1107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9812640601042e9974eff22d1646dfe: Generated new TSK 0
I20260810 18:58:12.431480 1107 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:58:12.463099 797 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.459731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46726 (local address 127.30.153.190:35507)
0810 18:58:12.459864 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.459869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.461122 (+ 1253us) server_negotiation.cc:407] Connection header received
0810 18:58:12.462109 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.462114 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.462171 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.462228 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.462957 (+ 729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46726: BlockingRecv error: recv got EOF from 127.0.0.1:46726 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 18:58:12.464617 1030 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.460493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.30.153.188:33963)
0810 18:58:12.460610 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.460614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.463492 (+ 2878us) server_negotiation.cc:407] Connection header received
0810 18:58:12.463827 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.463830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.463883 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.463937 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.464494 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47998: BlockingRecv error: recv got EOF from 127.0.0.1:47998 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 18:58:12.465902 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.461463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.30.153.189:44899)
0810 18:58:12.461586 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.461592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.464951 (+ 3359us) server_negotiation.cc:407] Connection header received
0810 18:58:12.465148 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.465151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.465200 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.465250 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.465784 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38716: BlockingRecv error: recv got EOF from 127.0.0.1:38716 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 18:58:12.472396 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.470806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38732 (local address 127.30.153.189:44899)
0810 18:58:12.470909 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.470913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.471450 (+ 537us) server_negotiation.cc:407] Connection header received
0810 18:58:12.471672 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.471675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.471753 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.471809 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.472299 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38732: BlockingRecv error: recv got EOF from 127.0.0.1:38732 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260810 18:58:12.477391 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.475965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.30.153.189:44899)
0810 18:58:12.476070 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.476074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.476536 (+ 462us) server_negotiation.cc:407] Connection header received
0810 18:58:12.476710 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.476713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.476760 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.476810 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.477294 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38734: BlockingRecv error: recv got EOF from 127.0.0.1:38734 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260810 18:58:12.483127 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.481657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.30.153.189:44899)
0810 18:58:12.481752 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.481755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.482192 (+ 437us) server_negotiation.cc:407] Connection header received
0810 18:58:12.482447 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.482451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.482500 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.482551 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.483036 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38736: BlockingRecv error: recv got EOF from 127.0.0.1:38736 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260810 18:58:12.490597 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.489184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.30.153.189:44899)
0810 18:58:12.489317 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.489321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.489747 (+ 426us) server_negotiation.cc:407] Connection header received
0810 18:58:12.489914 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.489917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.489960 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.490006 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.490507 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38738: BlockingRecv error: recv got EOF from 127.0.0.1:38738 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 18:58:12.502580 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.500778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.30.153.189:44899)
0810 18:58:12.500880 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.500884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.501427 (+ 543us) server_negotiation.cc:407] Connection header received
0810 18:58:12.501638 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.501641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.501698 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.501757 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.502311 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38744: BlockingRecv error: recv got EOF from 127.0.0.1:38744 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 18:58:12.523294 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.521366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.30.153.189:44899)
0810 18:58:12.521504 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.521508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.522116 (+ 608us) server_negotiation.cc:407] Connection header received
0810 18:58:12.522371 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.522374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.522484 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.522548 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.523165 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38754: BlockingRecv error: recv got EOF from 127.0.0.1:38754 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 18:58:12.559341 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.557950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.30.153.189:44899)
0810 18:58:12.558082 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.558087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.558376 (+ 289us) server_negotiation.cc:407] Connection header received
0810 18:58:12.558575 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.558584 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.558634 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.558726 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.559218 (+ 492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38762: BlockingRecv error: recv got EOF from 127.0.0.1:38762 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 18:58:12.626843 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.625599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38764 (local address 127.30.153.189:44899)
0810 18:58:12.625769 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.625774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.625934 (+ 160us) server_negotiation.cc:407] Connection header received
0810 18:58:12.626074 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.626077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.626126 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.626218 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.626690 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38764: BlockingRecv error: recv got EOF from 127.0.0.1:38764 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 18:58:12.758471 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:12.757104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.30.153.189:44899)
0810 18:58:12.757283 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:12.757288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:12.757429 (+ 141us) server_negotiation.cc:407] Connection header received
0810 18:58:12.757620 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:12.757625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:12.757686 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:12.757784 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:12.758295 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38766: BlockingRecv error: recv got EOF from 127.0.0.1:38766 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 18:58:13.018100 918 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:13.016869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.30.153.189:44899)
0810 18:58:13.017047 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:13.017052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:13.017096 (+ 44us) server_negotiation.cc:407] Connection header received
0810 18:58:13.017264 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:13.017267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:13.017318 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:13.017403 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:13.017953 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38780: BlockingRecv error: recv got EOF from 127.0.0.1:38780 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 18:58:13.348287 1105 catalog_manager.cc:1259] Loaded cluster ID: 8b109e83ce734a5d9bbc61c2565a5910
I20260810 18:58:13.348408 1105 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6: loading cluster ID for follower catalog manager: success
I20260810 18:58:13.350099 1105 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6: acquiring CA information for follower catalog manager: success
I20260810 18:58:13.351321 1105 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6a6453531bce4feabad062bcfccaf0c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:58:13.359710 1087 catalog_manager.cc:1259] Loaded cluster ID: 8b109e83ce734a5d9bbc61c2565a5910
I20260810 18:58:13.359793 1087 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95: loading cluster ID for follower catalog manager: success
I20260810 18:58:13.361555 1087 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95: acquiring CA information for follower catalog manager: success
I20260810 18:58:13.362893 1087 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf53bfc315694551b6725c68326c4d95: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:58:13.536821 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:13.532466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38792 (local address 127.30.153.189:44899)
0810 18:58:13.532763 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:13.532772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:13.532814 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:58:13.533022 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:13.533030 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:13.533169 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:13.533333 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:13.533631 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.534154 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.535115 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.535358 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.535477 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:13.535888 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:13.536352 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:13.536497 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:13.536570 (+ 73us) server_negotiation.cc:299] Negotiation successful
0810 18:58:13.536629 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
I20260810 18:58:13.543898 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:13.540034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48014 (local address 127.30.153.188:33963)
0810 18:58:13.540341 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:13.540347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:13.540364 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:58:13.540455 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:13.540457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:13.540563 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:13.540680 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:13.540940 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.541460 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.542320 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.542615 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.542726 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:13.543063 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:13.543468 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:13.543608 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:13.543669 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 18:58:13.543731 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":67,"threads_started":1}
I20260810 18:58:13.551456 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:13.547549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.30.153.189:44899)
0810 18:58:13.547693 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:13.547698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:13.547936 (+ 238us) server_negotiation.cc:407] Connection header received
0810 18:58:13.548120 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:13.548123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:13.548169 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:13.548227 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:13.548581 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.549214 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.550216 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.550503 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.550611 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:13.550933 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:13.551039 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:13.551107 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:13.551221 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 18:58:13.551296 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 18:58:13.551594 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:13.547316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.30.153.188:33963)
0810 18:58:13.547437 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:13.547441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:13.547453 (+ 12us) server_negotiation.cc:407] Connection header received
0810 18:58:13.547780 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:13.547784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:13.547847 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:13.547893 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:13.548229 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.548873 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.550132 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.550497 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.550579 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:13.551082 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:13.551184 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:13.551255 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:13.551404 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 18:58:13.551454 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 18:58:13.552305 1257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:13.546926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.30.153.190:35507)
0810 18:58:13.547175 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:13.547180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:13.547447 (+ 267us) server_negotiation.cc:407] Connection header received
0810 18:58:13.547546 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:13.547549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:13.547729 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:13.547861 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:13.548057 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.548778 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.550123 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:13.550505 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:13.550591 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:13.551166 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:13.551557 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:13.551653 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:13.552049 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 18:58:13.552115 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
I20260810 18:58:13.559993 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 682
I20260810 18:58:13.580623 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 800
I20260810 18:58:13.592785 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 919
2026-08-10T18:58:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3424 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 18:58:13 dist-test-slave-68xk krb5kdc[1264](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 18:58:13 dist-test-slave-68xk krb5kdc[1264](info): set up 2 sockets
Aug 10 18:58:13 dist-test-slave-68xk krb5kdc[1264](info): commencing operation
krb5kdc: starting...
W20260810 18:58:14.877674 31334 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/test-user.keytab.
Aug 10 18:58:14 dist-test-slave-68xk krb5kdc[1264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T18:58:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T18:58:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.153.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
Entry for principal kudu/127.30.153.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab.
I20260810 18:58:15.009975 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:44837
--webserver_interface=127.30.153.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:44837,127.30.153.189:40999,127.30.153.188:36649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 18:58:15.125281 1282 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:58:15.125725 1282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:15.125799 1282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:15.125885 1282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:15.131343 1282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:15.131403 1282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:15.131423 1282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:15.131440 1282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:15.131457 1282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:15.136837 1282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:44837,127.30.153.189:40999,127.30.153.188:36649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.190.keytab
--principal=kudu/127.30.153.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.190:44837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.153.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:15.137889 1282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:15.138947 1282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:15.147341 1292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:15.147360 1296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:15.147400 1289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:15.147624 1282 server_base.cc:1034] running on GCE node
I20260810 18:58:15.148298 1282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:15.149340 1282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:15.150498 1282 hybrid_clock.cc:648] HybridClock initialized: now 1786388295150481 us; error 37 us; skew 500 ppm
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:15.153263 1282 init.cc:377] Logged in from keytab as kudu/127.30.153.190@KRBTEST.COM (short username kudu)
I20260810 18:58:15.154071 1282 webserver.cc:460] Webserver started at http://127.30.153.190:40263/ using document root <none> and password file <none>
I20260810 18:58:15.154662 1282 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:15.154726 1282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:15.154897 1282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:15.156850 1282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/data/instance:
uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "cc0aa754f482f107b381f8eb52e162dc"
server_key_iv: "726afcdb896bc002a3f402b2c51bbf47"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.157358 1282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/wal/instance:
uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "cc0aa754f482f107b381f8eb52e162dc"
server_key_iv: "726afcdb896bc002a3f402b2c51bbf47"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.161119 1282 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 18:58:15.163523 1309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:15.164835 1282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 18:58:15.164937 1282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/wal
uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "cc0aa754f482f107b381f8eb52e162dc"
server_key_iv: "726afcdb896bc002a3f402b2c51bbf47"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.165017 1282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:15.178802 1282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:15.182026 1282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:15.182188 1282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:15.189808 1282 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.190:44837
I20260810 18:58:15.190135 1386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.190:44837 every 8 connection(s)
I20260810 18:58:15.190898 1282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/data/info.pb
I20260810 18:58:15.193890 1389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:15.197852 1389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.199031 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 1282
I20260810 18:58:15.199128 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-0/wal/instance
I20260810 18:58:15.199457 31334 external_mini_cluster.cc:1428] Setting key e6208d7edea8db2d99abd2c178cb48f6
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.189@KRBTEST.COM" created.
I20260810 18:58:15.215368 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.200286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:44837 (local address 127.0.0.1:37020)
0810 18:58:15.200899 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.200935 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.202061 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.202933 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.202939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.203364 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.203609 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.203622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.205414 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.205417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.206143 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.206151 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.207173 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.207177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.207306 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.207962 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.207981 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.209978 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.212052 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.212060 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.212067 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.212335 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.212787 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.212790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.212791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.212955 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.213616 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.213626 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.213825 (+ 199us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.214020 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.214487 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":89,"spinlock_wait_cycles":16640,"thread_start_us":167,"threads_started":1}
I20260810 18:58:15.215373 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.200285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37020 (local address 127.30.153.190:44837)
0810 18:58:15.200749 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.200768 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.201043 (+ 275us) server_negotiation.cc:407] Connection header received
0810 18:58:15.202502 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.202508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.202709 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.202865 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.203770 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.205272 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.206280 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.206984 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.207178 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.210142 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.210148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.210158 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.210181 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.211945 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.212467 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.212476 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.212478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.212669 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.213070 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.213073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.213074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.213471 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.213612 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.214109 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.214342 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":300,"thread_start_us":159,"threads_started":1}
I20260810 18:58:15.218456 1389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.219296 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.219021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42130)
0810 18:58:15.219151 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.219227 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 18:58:15.219544 1313 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
I20260810 18:58:15.220395 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.220181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42138)
0810 18:58:15.220286 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.220343 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 18:58:15.220561 1389 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:40999: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab.
Entry for principal kudu/127.30.153.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab.
I20260810 18:58:15.233711 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.189:40999
--webserver_interface=127.30.153.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab
--principal=kudu/127.30.153.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:44837,127.30.153.189:40999,127.30.153.188:36649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:58:15.269798 1389 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } attempt: 1
I20260810 18:58:15.270670 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.270348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42150)
0810 18:58:15.270522 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.270594 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 18:58:15.271502 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.271291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42156)
0810 18:58:15.271401 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.271447 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 18:58:15.271721 1389 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:40999: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
I20260810 18:58:15.339926 1389 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } attempt: 2
I20260810 18:58:15.340847 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.340541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42158)
0810 18:58:15.340697 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.340768 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 18:58:15.341671 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.341438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42174)
0810 18:58:15.341554 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.341610 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 18:58:15.341857 1389 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.189:40999: Network error: Client connection negotiation failed: client connection to 127.30.153.189:40999: connect: Connection refused (error 111)
W20260810 18:58:15.355299 1401 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:58:15.355561 1401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:15.355645 1401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:15.355746 1401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 18:58:15.359788 1401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:15.359843 1401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:15.359861 1401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:15.359911 1401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:15.359944 1401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:15.365222 1401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:44837,127.30.153.189:40999,127.30.153.188:36649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.189.keytab
--principal=kudu/127.30.153.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.189:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.153.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:15.366344 1401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:15.367394 1401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:15.376120 1412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:15.376129 1414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:15.376178 1410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:15.376353 1401 server_base.cc:1034] running on GCE node
I20260810 18:58:15.377071 1401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:15.378137 1401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:15.379304 1401 hybrid_clock.cc:648] HybridClock initialized: now 1786388295379266 us; error 54 us; skew 500 ppm
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:15.382126 1401 init.cc:377] Logged in from keytab as kudu/127.30.153.189@KRBTEST.COM (short username kudu)
I20260810 18:58:15.382982 1401 webserver.cc:460] Webserver started at http://127.30.153.189:34003/ using document root <none> and password file <none>
I20260810 18:58:15.383494 1401 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:15.383553 1401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:15.383713 1401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:15.385691 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/data/instance:
uuid: "c01b3f8909c54b229315f45ea65a37ca"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "39a0bde26617a4e18146fefd53ec2329"
server_key_iv: "c5524d2fd48221f094edfc66e231fef0"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.386246 1401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/wal/instance:
uuid: "c01b3f8909c54b229315f45ea65a37ca"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "39a0bde26617a4e18146fefd53ec2329"
server_key_iv: "c5524d2fd48221f094edfc66e231fef0"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.389971 1401 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260810 18:58:15.392432 1424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:15.393663 1401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 18:58:15.393819 1401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/wal
uuid: "c01b3f8909c54b229315f45ea65a37ca"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "39a0bde26617a4e18146fefd53ec2329"
server_key_iv: "c5524d2fd48221f094edfc66e231fef0"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.393931 1401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:15.411029 1401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:15.414335 1401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:15.414572 1401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:15.422176 1401 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.189:40999
I20260810 18:58:15.422518 1509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.189:40999 every 8 connection(s)
I20260810 18:58:15.423281 1401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/data/info.pb
I20260810 18:58:15.426148 1512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:15.430063 1512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.430915 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 1401
I20260810 18:58:15.431051 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-1/wal/instance
I20260810 18:58:15.431331 31334 external_mini_cluster.cc:1428] Setting key 138a97c84c3d8ecbab6cd4d779c60903
I20260810 18:58:15.433024 1389 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } attempt: 3
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.153.188@KRBTEST.COM; defaulting to no policy
I20260810 18:58:15.444666 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.433500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42188)
0810 18:58:15.433739 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.433757 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.433854 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.434954 (+ 1100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.434958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.435152 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.435285 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.435295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.436473 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.436479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.437326 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.437338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.438093 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.438097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.438197 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.438791 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.438809 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.440333 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.442560 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.442564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.442566 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.442938 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.443358 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.443361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.443363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.443455 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.444128 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.444131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.444230 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.444362 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.444500 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260810 18:58:15.445647 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.432169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37024 (local address 127.30.153.190:44837)
0810 18:58:15.432336 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.432340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.433380 (+ 1040us) server_negotiation.cc:407] Connection header received
0810 18:58:15.434619 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.434622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.434685 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.434785 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.435690 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.436494 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.437806 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.438955 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.439073 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.441996 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.442000 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.442003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.442028 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.443783 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.444345 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.444350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.444352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.444439 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.444718 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.444724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.444728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.444901 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.445008 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.445414 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.445524 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 18:58:15.445838 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.433890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.30.153.189:40999)
0810 18:58:15.434262 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.434275 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.434317 (+ 42us) server_negotiation.cc:407] Connection header received
0810 18:58:15.434501 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.434522 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.434782 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.434981 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.435385 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.436354 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.437463 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.437971 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.438142 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.440484 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.440490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.440499 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.440520 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.442449 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.443085 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.443093 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.443100 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.443249 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.443550 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.443555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.443557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.444015 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.444189 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.444506 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.444824 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"spinlock_wait_cycles":896,"thread_start_us":140,"threads_started":1}
I20260810 18:58:15.445845 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.432611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:44837 (local address 127.0.0.1:37024)
0810 18:58:15.433202 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.433267 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.434363 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.434842 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.434851 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.435324 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.435557 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.435567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.436623 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.436626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.437684 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.437692 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.439142 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.439149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.439291 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.439940 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.439966 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.441832 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.443939 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.443950 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.443955 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.444231 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.444535 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.444539 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.444541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.444622 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.445011 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.445020 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.445170 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.445324 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.445472 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"spinlock_wait_cycles":768,"thread_start_us":168,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.153.188@KRBTEST.COM" created.
I20260810 18:58:15.447486 1512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.448711 1389 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.449375 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.449130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48302)
0810 18:58:15.449241 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.449296 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 18:58:15.450140 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.449839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48316)
0810 18:58:15.450005 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.450064 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260810 18:58:15.450323 1389 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:36649: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
I20260810 18:58:15.457501 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.448624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.30.153.189:40999)
0810 18:58:15.448766 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.448773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.448790 (+ 17us) server_negotiation.cc:407] Connection header received
0810 18:58:15.448830 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.448834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.448885 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.448973 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.449874 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.450589 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.451806 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.452029 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.452105 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.454315 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.454319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.454323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.454348 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.455874 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.456322 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.456326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.456328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.456406 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.456699 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.456704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.456705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.456881 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.456983 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.457233 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.457382 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":46}
I20260810 18:58:15.457501 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.448541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42202)
0810 18:58:15.448664 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.448679 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.448755 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.449285 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.449288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.449609 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.449756 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.449767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.450699 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.450705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.451682 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.451697 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.452146 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.452153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.452294 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.452774 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.452794 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.454166 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.456005 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.456008 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.456011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.456203 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.456504 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.456506 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.456508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.456588 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.456984 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.456987 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.457075 (+ 88us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.457177 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.457322 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 18:58:15.458592 1512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.459362 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.459080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48330)
0810 18:58:15.459216 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.459300 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 18:58:15.459968 1428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
I20260810 18:58:15.460906 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.460714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48334)
0810 18:58:15.460811 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.460856 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 18:58:15.461130 1512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:36649: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.153.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab.
Entry for principal kudu/127.30.153.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab.
I20260810 18:58:15.465667 31334 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
/tmp/dist-test-taskBp6AcV/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.188:36649
--webserver_interface=127.30.153.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab
--principal=kudu/127.30.153.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.153.148:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.153.190:44837,127.30.153.189:40999,127.30.153.188:36649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 18:58:15.481495 1389 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } attempt: 1
I20260810 18:58:15.482317 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.481985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48336)
0810 18:58:15.482164 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.482228 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 18:58:15.483069 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.482823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48352)
0810 18:58:15.482934 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.482997 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 18:58:15.483203 1389 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:36649: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
I20260810 18:58:15.510430 1512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } attempt: 1
I20260810 18:58:15.511313 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.510997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48362)
0810 18:58:15.511154 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.511216 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 18:58:15.512096 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.511894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48376)
0810 18:58:15.511996 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.512042 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 18:58:15.512311 1512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:36649: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
I20260810 18:58:15.558382 1389 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } attempt: 2
I20260810 18:58:15.559288 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.558933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48388)
0810 18:58:15.559112 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.559186 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 18:58:15.560057 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.559815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48392)
0810 18:58:15.559936 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.559992 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 18:58:15.560194 1389 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:36649: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
W20260810 18:58:15.579043 1523 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 18:58:15.579310 1523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 18:58:15.579380 1523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 18:58:15.579473 1523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 18:58:15.580467 1512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } attempt: 2
I20260810 18:58:15.581344 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.580992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48402)
0810 18:58:15.581139 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.581199 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 18:58:15.582099 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.581909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48418)
0810 18:58:15.582006 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.582049 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 18:58:15.582269 1512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.153.188:36649: Network error: Client connection negotiation failed: client connection to 127.30.153.188:36649: connect: Connection refused (error 111)
W20260810 18:58:15.584092 1523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 18:58:15.584144 1523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 18:58:15.584163 1523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 18:58:15.584180 1523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 18:58:15.584196 1523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 18:58:15.589458 1523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.153.148:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.153.190:44837,127.30.153.189:40999,127.30.153.188:36649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/krb5kdc/kudu_127.30.153.188.keytab
--principal=kudu/127.30.153.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.153.188:36649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.153.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 10 Aug 2026 18:13:35 UTC on ccycloud.quasar-witmzp.root.comops.site
build id 4578678
I20260810 18:58:15.590523 1523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 18:58:15.591503 1523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 18:58:15.600502 1530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 18:58:15.600505 1531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:15.600854 1523 server_base.cc:1034] running on GCE node
W20260810 18:58:15.600502 1535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 18:58:15.601449 1523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 18:58:15.602741 1523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 18:58:15.603960 1523 hybrid_clock.cc:648] HybridClock initialized: now 1786388295603935 us; error 45 us; skew 500 ppm
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 18:58:15.607030 1523 init.cc:377] Logged in from keytab as kudu/127.30.153.188@KRBTEST.COM (short username kudu)
I20260810 18:58:15.607962 1523 webserver.cc:460] Webserver started at http://127.30.153.188:37497/ using document root <none> and password file <none>
I20260810 18:58:15.608551 1523 fs_manager.cc:359] Metadata directory not provided
I20260810 18:58:15.608646 1523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 18:58:15.608889 1523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 18:58:15.611131 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/data/instance:
uuid: "101f23c094fb44568ff4659158880903"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "20f3dccd550059b6363e80cfb5003013"
server_key_iv: "080a4a1eb978311388013161c0338baf"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.611709 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/wal/instance:
uuid: "101f23c094fb44568ff4659158880903"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "20f3dccd550059b6363e80cfb5003013"
server_key_iv: "080a4a1eb978311388013161c0338baf"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.615477 1523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 18:58:15.618937 1542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 18:58:15.620532 1523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 18:58:15.620680 1523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/data,/tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/wal
uuid: "101f23c094fb44568ff4659158880903"
format_stamp: "Formatted at 2026-08-10 18:58:15 on dist-test-slave-68xk"
server_key: "20f3dccd550059b6363e80cfb5003013"
server_key_iv: "080a4a1eb978311388013161c0338baf"
server_key_version: "encryptionkey@0"
I20260810 18:58:15.620808 1523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 18:58:15.642225 1523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 18:58:15.645181 1523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 18:58:15.645412 1523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 18:58:15.652704 1523 rpc_server.cc:307] RPC server started. Bound to: 127.30.153.188:36649
I20260810 18:58:15.652998 1637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.153.188:36649 every 8 connection(s)
I20260810 18:58:15.653709 1523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/data/info.pb
I20260810 18:58:15.656554 1638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 18:58:15.659345 1389 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } attempt: 3
I20260810 18:58:15.660737 1638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.663026 31334 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu as pid 1523
I20260810 18:58:15.663173 31334 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBp6AcV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786387990686100-31334-0/minicluster-data/master-2/wal/instance
I20260810 18:58:15.663512 31334 external_mini_cluster.cc:1428] Setting key 0ad9f6e77f2a739c1c14aae59f2a1a39
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.190@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
I20260810 18:58:15.673448 1512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } attempt: 3
I20260810 18:58:15.673403 1394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.659833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48424)
0810 18:58:15.659983 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.660001 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.660096 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.662473 (+ 2377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.662477 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.662732 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.662864 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.662872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.664850 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.664853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.665656 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.665667 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.666506 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.666510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.666591 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.667136 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.667152 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.668782 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.671268 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.671271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.671277 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.671533 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.671955 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.671960 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.671964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.672089 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.672834 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.672837 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.672934 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.673094 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.673237 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 18:58:15.674772 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.662599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37032 (local address 127.30.153.190:44837)
0810 18:58:15.662735 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.662739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.663556 (+ 817us) server_negotiation.cc:407] Connection header received
0810 18:58:15.663696 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.663702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.663744 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.663846 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.664811 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.665587 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.666742 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.667018 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.667139 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.670129 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.670133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.670137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.670166 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.671855 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.672352 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.672356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.672357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.672439 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.672750 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.672755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.672757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.673010 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.673089 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.674510 (+ 1421us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.674629 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":67}
I20260810 18:58:15.674885 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.660560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.30.153.188:36649)
0810 18:58:15.661019 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.661035 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.661072 (+ 37us) server_negotiation.cc:407] Connection header received
0810 18:58:15.661787 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.661804 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.662152 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.662485 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.663178 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.664682 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.665759 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.666338 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.666532 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.668939 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.668975 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.668985 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.669007 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.671145 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.671676 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.671682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.671690 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.671828 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.672214 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.672218 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.672220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.672746 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.672864 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.673265 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.673644 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":136,"threads_started":1}
I20260810 18:58:15.675711 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.664085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.30.153.188:36649)
0810 18:58:15.664415 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.664423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.664580 (+ 157us) server_negotiation.cc:407] Connection header received
0810 18:58:15.664678 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.664681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.664725 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.664819 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.665588 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.666391 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.667499 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.667853 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.667949 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.671128 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.671132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.671134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.671155 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.673738 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.674259 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.674264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.674267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.674355 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.674705 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.674708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.674710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.674933 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.675099 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.675312 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.675476 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":265,"spinlock_wait_cycles":6400,"thread_start_us":138,"threads_started":1}
I20260810 18:58:15.675736 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.662547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.190:44837 (local address 127.0.0.1:37032)
0810 18:58:15.663433 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.663473 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.663611 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.663898 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.663905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.664434 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.664662 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.664675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.665686 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.665688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.666603 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.666611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.667299 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.667304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.667417 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.667972 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.668015 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.669963 (+ 1948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.671964 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.671971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.671976 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.672232 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.672560 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.672563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.672565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.672646 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.674113 (+ 1467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.674119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.674233 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.675282 (+ 1049us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.675550 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":67,"spinlock_wait_cycles":896,"thread_start_us":97,"threads_started":1}
I20260810 18:58:15.677289 1638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.679423 1389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76: Bootstrap starting.
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.189@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
I20260810 18:58:15.682299 1389 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76: Neither blocks nor log segments found. Creating new log.
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
I20260810 18:58:15.683277 1389 log.cc:824] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76: Log is configured to *not* fsync() on all Append() calls
I20260810 18:58:15.686230 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.673963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48430)
0810 18:58:15.674253 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.674268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.674347 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.676220 (+ 1873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.676223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.676419 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.676527 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.676539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.677815 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.677820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.678652 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.678662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.679338 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.679341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.679423 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.679953 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.679967 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.681327 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.684337 (+ 3010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.684342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.684346 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.684717 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.685064 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.685067 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.685069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.685166 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.685642 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.685646 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.685757 (+ 111us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.685908 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.686041 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260810 18:58:15.686326 1389 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76: No bootstrap required, opened a new log
I20260810 18:58:15.686892 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.677721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.189:40999 (local address 127.0.0.1:42204)
0810 18:58:15.678164 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.678179 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.678260 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.678609 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.678612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.678777 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.678876 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.678882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.679718 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.679721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.680437 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.680448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.681139 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.681143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.681243 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.681657 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.681670 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.683029 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.685113 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.685118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.685121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.685371 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.685665 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.685668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.685670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.685754 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.686179 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.686182 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.686285 (+ 103us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.686580 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.686723 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374}
I20260810 18:58:15.686935 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.674527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48430 (local address 127.30.153.188:36649)
0810 18:58:15.675985 (+ 1458us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.675989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.676002 (+ 13us) server_negotiation.cc:407] Connection header received
0810 18:58:15.676034 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.676036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.676072 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.676645 (+ 573us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.676737 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.677678 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.678875 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.679199 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.679310 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.681726 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.681730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.681732 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.681764 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.684180 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.684831 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.684837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.684838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.684927 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.685306 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.685310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.685312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.685528 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.686551 (+ 1023us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.686620 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.686778 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":1398,"thread_start_us":120,"threads_started":1}
I20260810 18:58:15.687557 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.677806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42204 (local address 127.30.153.189:40999)
0810 18:58:15.677905 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.677909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.678266 (+ 357us) server_negotiation.cc:407] Connection header received
0810 18:58:15.678341 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.678346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.678394 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.678495 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.678976 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.679617 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.680570 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.680947 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.681069 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.683158 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.683164 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.683168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.683198 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.684984 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.685476 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.685480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.685481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.685564 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.685861 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.685864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.685866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.686050 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.686136 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.687342 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.687461 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 18:58:15.688407 1638 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } has no permanent_uuid. Determining permanent_uuid...
I20260810 18:58:15.690582 1512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca: Bootstrap starting.
I20260810 18:58:15.690634 1389 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } }
I20260810 18:58:15.690869 1389 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4538cf1dbae4baa9cb8f674f7a3dd76, State: Initialized, Role: FOLLOWER
I20260810 18:58:15.691210 1389 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } }
I20260810 18:58:15.691780 1647 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.692021 1647 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:15.692662 1389 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.153.188@KRBTEST.COM for kudu/127.30.153.188@KRBTEST.COM
I20260810 18:58:15.693403 1512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:15.694432 1512 log.cc:824] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca: Log is configured to *not* fsync() on all Append() calls
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.190@KRBTEST.COM
I20260810 18:58:15.696800 1512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca: No bootstrap required, opened a new log
I20260810 18:58:15.697039 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.688885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.153.188:36649 (local address 127.0.0.1:48438)
0810 18:58:15.689041 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 18:58:15.689052 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0810 18:58:15.689137 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 18:58:15.689351 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 18:58:15.689354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 18:58:15.689513 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0810 18:58:15.689571 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.689576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.690255 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.690261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.690961 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 18:58:15.690971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.691450 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.691457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 18:58:15.691556 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.692095 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 18:58:15.692114 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 18:58:15.693406 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 18:58:15.695550 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.695552 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.695554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.695777 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.696041 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 18:58:15.696043 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 18:58:15.696045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 18:58:15.696121 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 18:58:15.696549 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 18:58:15.696554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 18:58:15.696636 (+ 82us) client_negotiation.cc:769] Sending connection context
0810 18:58:15.696741 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 18:58:15.696870 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260810 18:58:15.697899 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.688914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.30.153.188:36649)
0810 18:58:15.689006 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.689010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.689119 (+ 109us) server_negotiation.cc:407] Connection header received
0810 18:58:15.689217 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.689219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.689252 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.689341 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.689679 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.690145 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.691084 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.691316 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.692649 (+ 1333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.693590 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.693595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.693598 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.693623 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.695437 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.695873 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.695876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.695878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.695951 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.696229 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.696232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.696233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.696421 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.696515 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.697592 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.697716 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 18:58:15.700651 1638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903: Bootstrap starting.
I20260810 18:58:15.701157 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.678863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37034 (local address 127.30.153.190:44837)
0810 18:58:15.683790 (+ 4927us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.683794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.683809 (+ 15us) server_negotiation.cc:407] Connection header received
0810 18:58:15.683850 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.683856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.683902 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.683983 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.684878 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.685626 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.686856 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.687148 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.687232 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.697374 (+ 10142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.697378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.697381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.697410 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.699079 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.699605 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.699609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.699611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.699696 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.699995 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.699998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.700000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.700207 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.700396 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.700819 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.700969 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4838}
I20260810 18:58:15.698114 1282 master_runner.cc:186] Error getting master registration for 127.30.153.189:40999: OK, instance_id { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" instance_seqno: 1786388295419947 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 18:58:15.702684 1282 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:58:15.703331 1638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903: Neither blocks nor log segments found. Creating new log.
I20260810 18:58:15.704309 1638 log.cc:824] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903: Log is configured to *not* fsync() on all Append() calls
W20260810 18:58:15.705744 1666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:15.705834 1666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:15.708014 1638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903: No bootstrap required, opened a new log
I20260810 18:58:15.708791 1512 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } }
I20260810 18:58:15.709039 1512 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01b3f8909c54b229315f45ea65a37ca, State: Initialized, Role: FOLLOWER
Aug 10 18:58:15 dist-test-slave-68xk krb5kdc[1264](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786388294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.153.189@KRBTEST.COM
I20260810 18:58:15.709399 1512 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } }
I20260810 18:58:15.711813 1512 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:15.712214 1638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } }
I20260810 18:58:15.712466 1638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101f23c094fb44568ff4659158880903, State: Initialized, Role: FOLLOWER
I20260810 18:58:15.712811 1638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } }
I20260810 18:58:15.713791 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.703038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.30.153.189:40999)
0810 18:58:15.703177 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.703184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.703214 (+ 30us) server_negotiation.cc:407] Connection header received
0810 18:58:15.703320 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.703323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.703377 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.703458 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.704345 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.705058 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.706155 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.706524 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.706607 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.709739 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.709745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.709752 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.709781 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.711635 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.712143 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.712148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.712151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.712257 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.712573 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.712576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.712579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.712827 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.713031 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.713445 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.713584 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 18:58:15.714156 1638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 18:58:15.718125 31334 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 18:58:15.720021 1347 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37034
I20260810 18:58:15.720248 1347 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 18:58:15.720361 1347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [term 0 FOLLOWER]: Advancing to term 1
W20260810 18:58:15.720741 1714 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:15.720816 1714 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:15.721661 1347 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } }
I20260810 18:58:15.722637 1347 leader_election.cc:288] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [CANDIDATE]: Term 1 election: Requested vote from peers c01b3f8909c54b229315f45ea65a37ca (127.30.153.189:40999), 101f23c094fb44568ff4659158880903 (127.30.153.188:36649)
I20260810 18:58:15.722594 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.722754 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:15.722956 1678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.723083 1678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:15.723908 1472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c01b3f8909c54b229315f45ea65a37ca"
I20260810 18:58:15.724129 1472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:15.724887 1401 master_runner.cc:263] No leader master found from master c01b3f8909c54b229315f45ea65a37ca
I20260810 18:58:15.724951 1401 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:58:15.725101 1523 master_runner.cc:263] No leader master found from master 101f23c094fb44568ff4659158880903
I20260810 18:58:15.725157 1523 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 18:58:15.725493 1472 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4538cf1dbae4baa9cb8f674f7a3dd76 in term 1.
I20260810 18:58:15.725929 1313 leader_election.cc:302] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4538cf1dbae4baa9cb8f674f7a3dd76, c01b3f8909c54b229315f45ea65a37ca; no voters:
I20260810 18:58:15.726233 1647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 18:58:15.726533 1647 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [term 1 LEADER]: Becoming Leader. State: Replica: b4538cf1dbae4baa9cb8f674f7a3dd76, State: Running, Role: LEADER
I20260810 18:58:15.726740 1598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "101f23c094fb44568ff4659158880903"
I20260810 18:58:15.726933 1598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [term 0 FOLLOWER]: Advancing to term 1
I20260810 18:58:15.726923 1647 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } }
I20260810 18:58:15.728256 1598 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4538cf1dbae4baa9cb8f674f7a3dd76 in term 1.
I20260810 18:58:15.728718 1647 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4538cf1dbae4baa9cb8f674f7a3dd76. Latest consensus state: current_term: 1 leader_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.728816 1647 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: This master's current role is: LEADER
W20260810 18:58:15.730356 1688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 18:58:15.730487 1688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 18:58:15.736227 1719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 18:58:15.737247 1719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 18:58:15.745990 1598 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [term 1 FOLLOWER]: Refusing update from remote peer b4538cf1dbae4baa9cb8f674f7a3dd76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:58:15.746768 1716 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:58:15.750723 1678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4538cf1dbae4baa9cb8f674f7a3dd76. Latest consensus state: current_term: 1 leader_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.750854 1678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:15.751797 1472 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [term 1 FOLLOWER]: Refusing update from remote peer b4538cf1dbae4baa9cb8f674f7a3dd76: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 18:58:15.752607 1716 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 18:58:15.756459 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4538cf1dbae4baa9cb8f674f7a3dd76. Latest consensus state: current_term: 1 leader_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.756598 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:15.762123 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.730719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42224 (local address 127.30.153.189:40999)
0810 18:58:15.730832 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.730836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.746486 (+ 15650us) server_negotiation.cc:407] Connection header received
0810 18:58:15.747889 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.747892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.747942 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.747992 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.748846 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.749639 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.750961 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.751308 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.751383 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.757875 (+ 6492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.757880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.757885 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.757918 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.759779 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.760529 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.760533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.760534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.760629 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.760970 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.760974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.760977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.761217 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.761344 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.761795 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.761916 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 18:58:15.770238 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.731358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.30.153.188:36649)
0810 18:58:15.731457 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.731461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.746888 (+ 15427us) server_negotiation.cc:407] Connection header received
0810 18:58:15.747246 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.747252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.747311 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.747397 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.755195 (+ 7798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.755920 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.759091 (+ 3171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.759454 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.759545 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.763168 (+ 3623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.763171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.763174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.763201 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.764877 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.767667 (+ 2790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.767670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.767672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.767765 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.768528 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.768532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.768533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.768757 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.768880 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.769944 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.770077 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260810 18:58:15.771133 1647 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.771279 1647 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:15.771454 1669 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.771590 1669 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:15.771720 1735 mvcc.cc:205] Tried to move back new op lower bound from 7317046459367636992 to 7317046459300900864. Current Snapshot: MvccSnapshot[applied={T|T < 7317046459367636992}]
I20260810 18:58:15.772157 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:15.730753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.30.153.190:44837)
0810 18:58:15.730871 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.730876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.746701 (+ 15825us) server_negotiation.cc:407] Connection header received
0810 18:58:15.747284 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.747287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.747336 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.747386 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.757031 (+ 9645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.757820 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.760079 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:15.760444 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:15.760525 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:15.765027 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 18:58:15.765030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 18:58:15.765053 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 18:58:15.765082 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 18:58:15.766807 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.768088 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.768092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.768094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.768189 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 18:58:15.769225 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 18:58:15.769228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 18:58:15.769230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 18:58:15.769459 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 18:58:15.769574 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:15.771868 (+ 2294us) server_negotiation.cc:299] Negotiation successful
0810 18:58:15.771998 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 18:58:15.772416 1739 mvcc.cc:205] Tried to move back new op lower bound from 7317046459367636992 to 7317046459300900864. Current Snapshot: MvccSnapshot[applied={T|T < 7317046459367636992}]
I20260810 18:58:15.772802 1716 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.772922 1716 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76 [sys.catalog]: This master's current role is: LEADER
I20260810 18:58:15.773999 1678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4538cf1dbae4baa9cb8f674f7a3dd76" member_type: VOTER last_known_addr { host: "127.30.153.190" port: 44837 } } peers { permanent_uuid: "c01b3f8909c54b229315f45ea65a37ca" member_type: VOTER last_known_addr { host: "127.30.153.189" port: 40999 } } peers { permanent_uuid: "101f23c094fb44568ff4659158880903" member_type: VOTER last_known_addr { host: "127.30.153.188" port: 36649 } } }
I20260810 18:58:15.774127 1678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 18:58:15.774628 1736 mvcc.cc:205] Tried to move back new op lower bound from 7317046459367636992 to 7317046459300900864. Current Snapshot: MvccSnapshot[applied={T|T < 7317046459367636992}]
I20260810 18:58:15.781641 1719 catalog_manager.cc:1347] Generated new cluster ID: acb62fe1bf3848e69c8281d6ad62858f
I20260810 18:58:15.781728 1719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 18:58:15.814745 1719 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 18:58:15.817183 1719 catalog_manager.cc:1504] Loading token signing keys...
I20260810 18:58:15.845736 1719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4538cf1dbae4baa9cb8f674f7a3dd76: Generated new TSK 0
I20260810 18:58:15.846583 1719 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 18:58:15.889235 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.885781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42240 (local address 127.30.153.189:40999)
0810 18:58:15.885931 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.885937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.887994 (+ 2057us) server_negotiation.cc:407] Connection header received
0810 18:58:15.888231 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.888235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.888293 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.888353 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.889077 (+ 724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42240: BlockingRecv error: recv got EOF from 127.0.0.1:42240 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 18:58:15.890888 1644 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.886715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48448 (local address 127.30.153.188:36649)
0810 18:58:15.886964 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.886969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.889792 (+ 2823us) server_negotiation.cc:407] Connection header received
0810 18:58:15.889981 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.889985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.890044 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.890102 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.890731 (+ 629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48448: BlockingRecv error: recv got EOF from 127.0.0.1:48448 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
W20260810 18:58:15.892370 1393 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.887304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.30.153.190:44837)
0810 18:58:15.887434 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.887439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.891449 (+ 4010us) server_negotiation.cc:407] Connection header received
0810 18:58:15.891622 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.891625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.891677 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.891727 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.892256 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37042: BlockingRecv error: recv got EOF from 127.0.0.1:37042 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 18:58:15.896842 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.895315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42250 (local address 127.30.153.189:40999)
0810 18:58:15.895415 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.895419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.895925 (+ 506us) server_negotiation.cc:407] Connection header received
0810 18:58:15.896124 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.896127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.896212 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.896263 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.896743 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42250: BlockingRecv error: recv got EOF from 127.0.0.1:42250 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 18:58:15.901799 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.900314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42266 (local address 127.30.153.189:40999)
0810 18:58:15.900411 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.900415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.900917 (+ 502us) server_negotiation.cc:407] Connection header received
0810 18:58:15.901088 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.901091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.901170 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.901231 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.901704 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42266: BlockingRecv error: recv got EOF from 127.0.0.1:42266 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260810 18:58:15.907464 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.905935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42270 (local address 127.30.153.189:40999)
0810 18:58:15.906033 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.906037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.906592 (+ 555us) server_negotiation.cc:407] Connection header received
0810 18:58:15.906777 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.906780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.906828 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.906877 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.907357 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42270: BlockingRecv error: recv got EOF from 127.0.0.1:42270 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 18:58:15.916535 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.914800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42282 (local address 127.30.153.189:40999)
0810 18:58:15.914941 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.914945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.915546 (+ 601us) server_negotiation.cc:407] Connection header received
0810 18:58:15.915722 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.915725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.915773 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.915822 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.916431 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42282: BlockingRecv error: recv got EOF from 127.0.0.1:42282 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 18:58:15.928670 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.927256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.30.153.189:40999)
0810 18:58:15.927391 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.927396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.927671 (+ 275us) server_negotiation.cc:407] Connection header received
0810 18:58:15.927863 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.927869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.927920 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.928024 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.928529 (+ 505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42288: BlockingRecv error: recv got EOF from 127.0.0.1:42288 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 18:58:15.948206 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.946792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.30.153.189:40999)
0810 18:58:15.946930 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.946935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.947208 (+ 273us) server_negotiation.cc:407] Connection header received
0810 18:58:15.947401 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.947404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.947467 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.947566 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.948059 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42298: BlockingRecv error: recv got EOF from 127.0.0.1:42298 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 18:58:15.983580 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:15.982302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42302 (local address 127.30.153.189:40999)
0810 18:58:15.982501 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:15.982505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:15.982679 (+ 174us) server_negotiation.cc:407] Connection header received
0810 18:58:15.982752 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:15.982755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:15.982813 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:15.982909 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:15.983424 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42302: BlockingRecv error: recv got EOF from 127.0.0.1:42302 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260810 18:58:16.051142 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:16.049905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42316 (local address 127.30.153.189:40999)
0810 18:58:16.050071 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:16.050075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:16.050192 (+ 117us) server_negotiation.cc:407] Connection header received
0810 18:58:16.050269 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:16.050272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:16.050331 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:16.050516 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:16.050980 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42316: BlockingRecv error: recv got EOF from 127.0.0.1:42316 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 18:58:16.182862 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:16.181392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42330 (local address 127.30.153.189:40999)
0810 18:58:16.181534 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:16.181538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:16.181815 (+ 277us) server_negotiation.cc:407] Connection header received
0810 18:58:16.181936 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:16.181939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:16.181993 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:16.182071 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:16.182673 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42330: BlockingRecv error: recv got EOF from 127.0.0.1:42330 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260810 18:58:16.442094 1519 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 18:58:16.440906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.30.153.189:40999)
0810 18:58:16.441083 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:16.441087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:16.441103 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:58:16.441235 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:16.441237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:16.441287 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:16.441360 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 18:58:16.441931 (+ 571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42342: BlockingRecv error: recv got EOF from 127.0.0.1:42342 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 18:58:16.722260 1714 catalog_manager.cc:1259] Loaded cluster ID: acb62fe1bf3848e69c8281d6ad62858f
I20260810 18:58:16.722349 1714 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903: loading cluster ID for follower catalog manager: success
I20260810 18:58:16.724141 1714 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903: acquiring CA information for follower catalog manager: success
I20260810 18:58:16.725281 1714 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 101f23c094fb44568ff4659158880903: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:58:16.731702 1688 catalog_manager.cc:1259] Loaded cluster ID: acb62fe1bf3848e69c8281d6ad62858f
I20260810 18:58:16.731771 1688 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca: loading cluster ID for follower catalog manager: success
I20260810 18:58:16.733474 1688 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca: acquiring CA information for follower catalog manager: success
I20260810 18:58:16.734678 1688 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c01b3f8909c54b229315f45ea65a37ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 18:58:16.960469 1855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:16.956306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42346 (local address 127.30.153.189:40999)
0810 18:58:16.956564 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:16.956569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:16.956685 (+ 116us) server_negotiation.cc:407] Connection header received
0810 18:58:16.956868 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:16.956872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:16.957022 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:16.957149 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:16.957428 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.957965 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.958926 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.959155 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.959268 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:16.959610 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:16.960046 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:16.960186 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:16.960247 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 18:58:16.960306 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260810 18:58:16.967269 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:16.963515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.30.153.188:36649)
0810 18:58:16.963813 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:16.963818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:16.963834 (+ 16us) server_negotiation.cc:407] Connection header received
0810 18:58:16.963916 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:16.963918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:16.964027 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:16.964125 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:16.964306 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.964817 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.965665 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.965893 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.965986 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:16.966258 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:16.966760 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:16.966858 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:16.966993 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 18:58:16.967107 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":96,"threads_started":1}
I20260810 18:58:16.974397 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:16.970646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.30.153.188:36649)
0810 18:58:16.970748 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:16.970752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:16.970905 (+ 153us) server_negotiation.cc:407] Connection header received
0810 18:58:16.971236 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:16.971239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:16.971286 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:16.971356 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:16.971690 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.972353 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.973211 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.973551 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.973688 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:16.973916 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:16.973992 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:16.974060 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:16.974209 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 18:58:16.974272 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 18:58:16.974766 1855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:16.970186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.30.153.189:40999)
0810 18:58:16.970324 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:16.970330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:16.970871 (+ 541us) server_negotiation.cc:407] Connection header received
0810 18:58:16.970991 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:16.970997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:16.971053 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:16.971124 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:16.971395 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.972065 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.973313 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.973636 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.973727 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:16.974090 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:16.974195 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:16.974268 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:16.974561 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 18:58:16.974621 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 18:58:16.975530 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 18:58:16.970140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37058 (local address 127.30.153.190:44837)
0810 18:58:16.970392 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 18:58:16.970396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 18:58:16.970587 (+ 191us) server_negotiation.cc:407] Connection header received
0810 18:58:16.970747 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 18:58:16.970751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 18:58:16.970928 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 18:58:16.971053 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 18:58:16.971233 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.971934 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.973335 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 18:58:16.973558 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 18:58:16.973670 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 18:58:16.974390 (+ 720us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 18:58:16.974966 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 18:58:16.975058 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 18:58:16.975328 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 18:58:16.975382 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":101,"threads_started":1}
I20260810 18:58:16.984383 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 1282
I20260810 18:58:17.006112 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 1401
I20260810 18:58:17.019646 31334 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBp6AcV/build/debug/bin/kudu with pid 1523
2026-08-10T18:58:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3424 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6849 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (306328 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST